How will Ground Truth Colour Standardisation yield the benefits of AI in Medical Imaging?
Posted 6th November 2019 by Joshua Sewell
The essence of colour management is to ensure that the original target translates through a digital pathway so that the output images are exactly the same colour as the original.
When applied to medical imaging, and considering all the specific stains used in pathology, colour management becomes important. Particular coloured stains bind specific structures of cells in tissue to confer visualisation of diagnostic information. Without translating the colour through the digital pathway correctly, you lose the aspect of diagnostic information, which comes from specific colours.
What is Ground Truth Colour Standardisation?
Medical ‘ground truth’ is analysis from direct observation of the original raw data for the sample in question. For example, in tissue pathology, the ground truth data is the pathological information contained in the section of stained tissue.
There are three basic components of information within the tissue. Firstly, shape data comprises a cell or group of cells. Secondly, spatial data comprises the relationship between those cell shape-data and all the cells in the tissue. Thirdly, the actual colour used to visualise the shape and spatial data. If we took this down to a pixel level such as in Deep Learning on images, colour begins to form a large part of each pixel’s data, before spatial or even shape analysis.
Therefore, when we talk about ground-truth colour we’re talking about the real colour in a tissue sample and making sure that information is accurately passed to a pathologist or AI algorithm. This enables AI to make decision based on 100% reality, bringing diagnosis closer to that of humans.
How could Colour Management combine with Artificial Intelligence?
Colour management can have a big impact on the use of AI in pathology in two ways.
- Use of Multiple Scanners by Standardisation of Final Images.
A digital scanner is an amalgamation of different optical components, each introducing its own variation along the digital imaging path. A lab may use multiple scanners from different manufacturers and each of those scanners is made of different components and therefore introduce different amounts of colour variation into the final data.
When comparing two scanners, often even the human eye can see that there is a large difference in the interpretation of the colour. AI on pixel-level data tries to find patterns and differences between data. Therefore, using images from two different scanners introduces digital variation that is not representative of the decisions made on ground-truth data.
Performing AI on datasets formed from multiple scanners requires standardisation to ground truth to ensure data uniformity and reliable AI.
- Calibration and Validation
A scanner can vary in its final output as components age with use or are replaced during service visits, and therefore needs calibration. If an uncalibrated scanner is used through big datasets, for example, there could be colour changes between sample 1 and sample 1,000.
Colour management offers a way to ensure that when you have to analyse your data from the same, or different, scanners over time. The actual data collection is validated, beginning to end. Therefore, it is possible to ensure that colour digitisation is not a caveat when performing analysis at the end, human or AI.
How is colour management being done at the moment?
There are three main methods for WSI colour management.
- The digital photography option.
There is a physical option which is somewhat like a high-end digital camera. You can calibrate a WSI device to a selection of colours on a piece of material, such as paper, to result in the images using colour values representative of what is on the paper. This technique is often used in digital photography, where calibration software is also built-in to the camera.
A problem with this method is the occurrence of metamerism. This is where spectral responses are not the same as the human eye interprets. For example, white paper has fluorescent properties which we don’t necessarily see. If you were to put red ink on white paper and then put a red histology dye on a piece of tissue, it might look the same to the naked eye. However, once digitised by the scanner, there are two different spectral responses, which introduces errors via this method.
- The AI Route
Another approach uses artificial intelligence in an averaging activity; performing AI that takes colour data from one type of scanner to transform colour from one scanner output to a different scanner output across large, mixed datasets.
This has its own problems because in reality this actively changes medical data from one output to a different output. This means you might change the diagnostic information itself. Equally, as all scanners have inherent colour errors, then standardising the data to one scanner type essentially makes the final colours equally wrong, rather than correct when compared to the original tissue. Other methods universally apply aspects of hue and saturation from guideline values, to similar effect – simulation of expected colour, not the actual and real colour.
- External Device
The best method uses an external slide-based device, compatible with WSI digital pathology, but removing metamerism errors by incorporation of histology stains onto material that mimics binding to biological tissue. In this case, you can use the true biological colour in real scanning settings. This device is therefore universal for use on any make or model of scanner, and you can standardise the colour output across all WSI devices, all based upon real, ground-truth colour.
What does the future hold for colour management, pathology and AI?
Colour management in digital pathology is a growing topic, which is unsurprising when looking at the diagnostic landscape compared to ten years ago. As can be seen from the 6th Digital Pathology & AI Congress: Europe, there is a massive market around this digitisation. The more digital pathology becomes commonplace, the greater the need to address the uniquely digital caveats that accompany the technology.
In recognition of this, influential bodies such as the FDA in the US, and to some degree the Royal College of Pathologists in the UK, have issued guidelines. These suggest that when it comes to the topic of colour management, there should be validation of colour across devices, as well as appropriate methods to validate and calibrate the devices to ensure consistency.
Therefore, we see colour management performing a key regulatory and quality assurance role in the future. With this growing market and an environment of labs digitising their historic and future samples, the medical processes of using digital imagery in medical diagnosis are already there. Alongside this, the regulatory implications of devices producing medical data that is being used in life-critical decisions make it important that colour management of inherent errors is considered going forward.
The positive implication of colour management is the ability to ensure that digital images are ground-truth colour. This means that they contain all the pathological information represented in the colour of the tissue sample that is made available through a light microscope. Another is the ability to standardise the output from scanners to ground-truth colour so that any image from any scanner is 100% correct. This fits into a growing demand for end-to-end validation and solves a modern digital issue additional to established pathology.
Therefore, the implication for artificial intelligence is that there is a reduced chance that AI is misdirected, by even a small amount – even if colour data is only a small detail of a single image, if AI incorrectly learns from or misses these details over tens of thousands of images, it may have drastic impact downstream.
All in all, ground-truth colour is important part of digital pathology, and standardising and validating scanner output is needed across the board for better diagnostics.
Leave a Reply