A paper recently published in The Lancet by Leibig et al.(2022); looks at the effectiveness of an AI system to detect breast cancer on mammograms, reports our Medical Director, Dr Paul Zollinger-Read CBE.
What study has been done to investigate the detection of breast cancer with AI?
The study took place in Germany and involved just over 82,000 women. Standard practice is for two radiologists to examine each mammogram and, if they detect suspicious findings, the women are called for further investigations such as a biopsy.
Previous studies have demonstrated that AI tools can reliably detect the signs of breast cancer on mammograms; however, Liebig and colleagues wanted to investigate the most effective way to use these tools.
How were the AI tool tests conducted?
They compared the performance of the AI tool in two different situations; firstly, directly replacing the radiologist and then secondly, when the tool is used as a decision support tool by a radiologist.
In the first situation, when used as a stand-alone tool, it performed slightly below the level of the radiologist.
When used as a decision support tool, the combination of AI and a radiologist performed better than the radiologist alone.
What does this breast study and AI study show?
What this study also shows is that each AI system is unique. AI tools are developed by “exposing” them to many mammograms, and each tool will have unique characteristics.
The effectiveness of such tools is often measured by their sensitivity and specificity. Some will be very sensitive; that is, they are good at detecting disease.
Others will have lower sensitivity, so they are not so good at detecting the disease.
However, detecting disease is one aspect; we also need the test to reliably detect when the condition isn’t present. This is measured by the specificity.
Some tests may have high specificity, which means they can reliably be confident that no disease is present.
However, others are not so ‘specific’, Which, in health care terms means that they may identify some false positives. ie., it says you have the disease when you do not.
Therefore, the usefulness of these tools depends upon the balance of their sensitivity and specificity, which will be unique for each tool and the situation it is used in.
How is AI being used in healthcare?
This paper is very much in line with the direction of AI in health care at the moment, where it is seen more as a support for clinicians in decision-making rather than a replacement.