In this article, Dr Nalini Bhola from BreastScreen NSW and Associate Professor Helen Frazer from BreastScreen Victoria introduce their respective state’s technological solutions for improving the accuracy and efficiency of cancer detection in mammography screening.
Breast cancer is the most common cancer in women in Australia, and routine free mammograms are available to women over 40 to reduce mortality risk through early detection. Associate Professor Frazer states, “Breast cancer is only associated with deaths if it spreads beyond the breast. So, the whole premise is to get cancer when it’s early”. In the traditional blind double-reader process, two radiologists will give their independent assessment of an image and if their opinions differ, a third – typically more senior – radiologist will adjudicate to clear or recall the patient.
But the gap between clinical demand and supply of radiologists has raised concerns about the sustainability of breast screening programs – with risks of longer delays before cancer diagnosis, increasing client wait times, and a higher chance of radiologist burnout.
Six months ago, in November 2024, BreastScreen NSW became Australia’s first population-based screening program to introduce machine reading technology to support radiologists in interpreting a select range of mammography images. Using an existing AI product tested in a world-first study of over 650,000 mammograms, the organisation now uses the AI reader in a radiologist-AI combined approach to screen a cohort of women. In this interview, Dr Bhola provides insights into the strong clinical governance used during the evaluation and through implementation, the minimal workflow changes for patients and clinicians, and the potential for the technology to support the radiologist workforce.
In Victoria, Associate Professor Helen Frazer discusses how BreastScreen Victoria have taken a different AI implementation approach, with an emphasis on developing local sovereign capabilities with our globally unique breast health datasets for AI development and benchmarking. With its partners, BreastScreen Victoria have created a state-of-the-art mammogram AI reader and breast cancer risk score – with the AI reader implemented in a randomised controlled trial (RCT) this year. Associate Professor Frazer describes a fundamental future shift, where every time a woman is screened, they can be advised of their personalised 4-year risk derived using AI and provided an individualised screening prescription with the potential to save more lives.
National AI reform is, however, not solely the responsibility of the federal government. The states have a clear role in health care service delivery which requires digital infrastructure, and which will increasingly use AI to deliver services clinically and back-of-house. Industry has a clear role, as do the health professions. Indeed, the AAAiH was created because we recognised that AI reform is complex and whole of nation, and not just a federal government obligation. We have consistently asked for a national AI in Healthcare Council to bring these disparate players together. But it is much harder to do without the feds. We can only hope that the next 2 months see considered and bold policy in the AI space that specifically addresses the many unique safety and quality challenges we find in healthcare.
Dr Nalini Bhola, BreastScreen NSW
What clinical problem was BreastScreen NSW looking to solve using AI?
There is a well-reported global shortage of radiologists, especially breast radiologists. BreastScreen NSW is among an increasing number of international health agencies introducing machine reading technology as a complement to our expertise as radiologists. AI is not going to solve our workforce shortages, but it can support our workforce to use their time optimally.
What is the AI system?
We use a commercially available Therapeutic Goods Administration (TGA) approved computer image recognition technology that supports our radiologists to interpret mammographic images. Its defined algorithm ensures outcomes stay consistent. We began evaluating the off-the-shelf system in 2023, given our diverse Australian population and to ensure Indigenous women have appropriate outcomes. The technology produced comparable results to our existing programme and didn’t compromise cancer detection rates. We successfully implemented machine reading across BreastScreen NSW in November 2024. Competitive procurement was used to ensure probity and value for money. The project took 5 years to reach implementation, covering initiation, approval, commencement, product selection, and evaluation of the product from 2023.
What have been the greatest challenges from using the AI screening tool?
Being the first population breast screening program in Australia has been challenging, given the enormity of our [evaluation] study. We screened over 650,000 digital mammographic examinations from BreastScreen NSW, looking at the cancer detection rates of AI-radiologist readings compared to radiologist-only readings. This included a cohort of almost 4,500 detected cancers and a further 1,171 interval cancers.
System integration was also a hurdle. We have a Business Information System (with information about the case and clinical history) and a Picture Archiving System (to view the images). These systems need to integrate with the AI, whilst ensuring that data is de-identified, so we don’t compromise security or privacy. With the help of the vendors of those systems and our colleagues in eHealth NSW we now have a fully integrated and fully-automated system, with very robust privacy and cybersecurity controls.
What financial or time costs came with the implementation?
To give our BreastScreen clinicians the confidence needed to accept the AI reader, we led them through the process and evaluation results and gave the opportunity to ask questions. We addressed data security risks and conducted relevant training. Regarding our clients, we prioritised transparency, providing posters in the screening space and consent forms to ensure they were comfortable. The study too was costly, but we learnt that it would be cost-effective in the future, safe for women, and secure from a data perspective.
What workflow changes came with AI implementation?
The workflow implications were minimal for women coming in, they may receive their results faster. From a clinician perspective, there’s little change in workflow too, so it’s positive that it’s not impacting workflows.
What clinical governance structures do you have in place use to ensure that the system is safe for use?
Our Clinical Advisory Group guided our strong governance during the evaluation and through implementation. Our steering committee oversaw several working groups, covering Epidemiology, Clinical Advice, Technical Support, and Communication and Change Management. We’re also taking a conservative implementation approach at this stage, so just a proportion – about 31,000 mammograms out of 370,000 women screened annually – get a machine read. Established quality and governance groups within BreastScreen NSW review performance regularly and ensure that the technology continues to perform as expected.
What are the greatest benefits that you’ve seen from using AI?
We’re seeing improvements in client experience and outcomes. Less women are needing to come back to the clinic whilst clinical safety and quality is still maintained, due to the optimised, lower recall rate of the machine reader. We hope because of this, the radiologist can focus more time on the complex clinical workload.
What advice do you have for another organisation looking to adopt an AI tool?
To confirm the AI technology is safe and effective, ensure the technology is tested against proposed and current clinical settings. This includes consulting widely, addressing client confidentiality and privacy, legal compliance, and continual documentation, monitoring, and evaluation. Address clinical governance during evaluation and what it’s going to look like beyond. Regarding workflows and clinical acceptance, I’ve found that constant and transparent communication is important to support our clinicians and for external stakeholders.
Associate Professor Helen Frazer, BreastScreen Victoria
What clinical problem was BreastScreen Victoria looking to solve using AI?
Breast cancer screening is a highly successful initiative with proven survival benefits. But it is a one-size-fits-all programme with challenges in accuracy, experience, and costs. There can be unnecessary recalls – the false positives – where a woman is recalled into the assessment clinic pathway with an indication for cancer but is subsequently determined to not have cancer. The false negatives are the interval cancers found in the interval between regular screening mammograms. These contribute to the harms and costs of the programme.
The client’s experience is another challenge. The programme is not personalised based on a woman’s risk level and it’s less suited to younger women below 50 with dense breasts, which can both camouflage and increase a woman’s risk of developing breast cancer. The final challenges are efficiencies, costs, and programme participation and experience. Results are slow against today’s expectations – it takes up to 14 days to receive an all-clear result. We have increasing demands with our ageing population, an increasing incidence of breast cancer, and challenges with having enough radiologists to meet demand. We set out for a triple benefit – to see how we can use AI with our radiologists to improve on accuracy and save lives, to personalise the screening experience, and to improve the productivity of the programme.
What is the AI system?
As background, BreastScreen Victoria in collaboration with our research partners have built our own globally unique longitudinal datasets with strong ground truths, local capabilities to develop, train, validate and benchmark algorithms, retrospective simulations, and prospective capabilities to test the human-AI interaction. We’re about to run one of the first RCTs globally that uses AI in clinically important decisions. We’ve developed the BRAIx AI reader, a state-of-the-art mammogram reader that can result in better accuracy, client experience, and efficiency for the system. We’ve developed a globally unique risk score, generated by the AI reader, that predicts cancer risk over the next four years better than any other known method. We are also benchmarking algorithms developed on overseas populations.
What have been the greatest challenges from using the AI screening tool?
The first challenge to consider is building the datasets. What I’ve learnt is there’s nothing particularly unique with the algorithm software. All the breast cancer detection algorithms, whether it be BreastScreen Victoria (and partners) developed BRAIx AI reader or those that are commercially available, performed similarly if trained on high quality datasets. So, the real value is in our local datasets for training and testing – and we’ve spent a lot of time building these high-quality population-scale datasets with strong ground truths and the scale required to develop and test for bias and generalisability across all segments of our population. This is especially important in our minority cohorts, for instance our Aboriginal and Torres Strait Islander women, to ensure that AI is safe and fair for all.
Another challenge was winning support for local development. Relying on privately-owned and international algorithms places sovereign risks into our health system and risks hollowing out important local clinical and technical capabilities for our algorithmic future. This debate around build-versus-buy is a bigger discussion that we need to have as a local health community as AI tools emerge for clinical decision-making.
What financial or time costs come with AI implementation?
There are major costs in dataset development, algorithmic development and testing, setting up for a large-scale RCT, and final integration into clinical workflows. If you think of the whole lifecycle, the costs are significant. We’ve been fortunate because the BRAIx programme is supported by funding sources such as the Commonwealth Medical Research Future Fund that believe in our Australian clinical expertise and in supporting sovereign data and AI capability.
What workflow changes come with AI implementation?
Breast cancer screening is an exemplar use case because workflow changes are minimal in the first stage of implementation. We’re replacing one of the two radiologist readers, so a radiologist is still in charge of every single decision. You won’t find any other areas of radiology where every image is read by two and sometimes three radiologists. In Victoria, we’re doing that through an RCT across our population.
We now envisage a fundamental shift towards personalised risk-based screening enabled by the AI risk score. Unlike the current model, where a radiologist says “Yes” or “No” and if its “No” the client is rescreened in two years, we have discovered that the AI algorithm can produce a score of your likely risk of breast cancer in the next 4 years that is more accurate than any known methods. We’re about to publish a paper demonstrating the use of the AI reader in predicting 4-year cancer risk and envision a path towards risk-based stratification based on an automated risk score derived from the mammogram.
What clinical governance structures do you have in place to ensure that the system is safe for use?
We have an existing operating first-line quality control at screening services and second-line program governance with key operating metrics, national standards, and independent audit and accreditation processes embedded at BreastScreen Australia. The RCT has also multiple levels of governance to ensure safety – via Human Research Ethics Committee approval, an independent data safety and monitoring committee, and state-wide monitoring of early screening indicators.
What are the greatest benefits that you’ve seen from using AI?
We’re about to start the RCT and from our retrospective simulation study, we anticipate increased cancer detection, lower unnecessary recalls, faster time to results, and assistance with workload challenges. When we looked at our AI reader’s performance on known false negatives, we found that it detected up to 40% of them, which is significant because the false negatives are the most distressing aspect of what we do. AI is positive in terms of reducing those, even in women with dense breasts. Alongside reducing false negatives, we can also reduce unnecessary assessments (false positives) and speed up the time to provide results. When I talk to women, they find that two-week wait really stressful.
What advice do you have for another organisation looking to adopt an AI tool?
Don’t dismiss what we can develop here in Australia. It is the quality of data that matters the most – and Australia has some of the highest quality datasets globally. The breast cancer population screening programme data has been developed with public funding over 30 years; it’s an incredible resource. We must consider data and AI sovereignty and engage in those build-versus-buy debates as a community. It’s important that our clinical and technical capabilities aren’t hollowed out such that we become dependent on black boxes for high-consequence decisions or have the sovereign risks of using a single international provider. We can do a lot. The algorithms are pretty non-unique; it’s the data that’s unique.
Dr Nalini Bhola is a radiologist with subspecialist qualifications in breast imaging. She is the State Clinical Director for BreastScreen NSW and provides clinical advice and strategic leadership, sets clinical standards, and supports quality assurance and improvement activities within the program. Based in regional NSW, she also provides diagnostic breast imaging services in private practice.
Associate Professor Helen Frazer is a radiologist, breast cancer clinician and AI researcher with over 25 years’ experience in breast cancer screening services. She is State Clinical Director for BreastScreen Victoria, Clinical Director for St Vincent’s BreastScreen and Associate Professor at the University of Melbourne. She is Lead Investigator in multiple AI developments including the BRAIx “Transforming Breast Cancer Screening with AI” programme – an MRFF-funded partnership comprising BreastScreen Victoria, St Vincent’s Institute of Medical Research, St Vincent’s Hospital Melbourne, University of Melbourne, and Australian Institute of Machine Learning at University of Adelaide.