Distinction of Benign and Malignant Lesions through Deep Learning

Posted 6th August 2018 by Jane Williams

Machine learning is already prevalent in many industries and most pathologists are unaware how accessible machine learning is and how it can be used to augment their work or research. Applications include decision support, image analytics, process improvement, disease diagnosis and prognosis.

The basic idea behind machine learning is to use a machine learning algorithm to learn the pattern behind a dataset, from which predictions can be made on an unknown set of similarly formatted data. The dataset could involve various types of data, including laboratory values or features of pathology related images. Various open-source and commercial software for machine learning and image analytics are currently available, many of which were showcased at the Digital Pathology & AI Congress in New York City. Some open-source tools for machine learning I have used include R, python with scikit-learn, Knime, and Orange. Knime and Orange use a drag and drop type of user-interface and are great for anyone who are not adept at computer programming.

In this article, I will be focusing on a type of machine learning referred to as Convolutional Neural Network (CNN). It’s a deep learning method inspired by the way our visual cortex processes images through receptive fields whereby individual retinal neurons receive stimuli from different regions of our visual field and information from multiple retinal neurons are later passed on to neurons further down the chain.

Likewise, CNN has a feed-forward neural network architecture that relies on multiple layers of convolution and pooling, followed by one or more fully connected layers. Convolution or pooling operations are carried out on information from one layer and the results are passed on to a deeper layer of the network. CNN is frequently used on image datasets although it also has applications in natural language processing. You can build your CNN image classification model from scratch through deep learning frameworks like Tensorflow or use a pre-built CNN architecture like VGG-16 and Inception V3.

Given the right amount of training data and a proper CNN architecture, one often can get results with impressive accuracy. Recently, CNN has been gaining popularity in the field of Digital Pathology and it is interesting to note a Pubmed search for “Convolutional Neural Network” and “Pathology” retrieved close to 100 articles, the majority of which were published in the past 3 years.

As an exercise, I recently took 144 images (72 benign + 72 malignant) from the paper “Pigeons (Columba livia) as trainable observers of pathology and radiology breast cancer images” and extracted 4096 features from each image using VGG-16 with Orange (software) as the machine learning platform. The 4096 feature variables are derived through VGG-16 by running an image through VGG-16 and stripping the last fully connected layer from the model. Models were built with the data and trained using several machine learning algorithms such as logistic regression, SVM, and random forest.

Evaluation through 10-fold cross-validation and random sampling using a training set size of 70% yielded ROC AUC scores ranging from 0.888 to 0.96 and classification accuracies ranging from 0.819 to 0.91. For comparison, in the original paper from which the images were procured, a single Pigeon trained on one subset of the images and tested on another subset managed to achieve a classification accuracy of 0.85. A similarity grid based on multidimensional scaling can be used to illustrate the feature differences between malignant and benign images. Even with a limited dataset, the signals derived from the VGG-16 features were sufficient to show a transition from benign to malignant images:

Additionally, dimensional reduction on the VGG-16 feature set using t-SNE (t-distributed stochastic neighbour embedding) with cosine distances showed separate clusters for benign (blue) and malignant (red) images:

The VGG-16 model used in the experiment was pre-trained on the ImageNet image database. It is amazing it performed as well as it did considering none of the ImageNet images were based on histological material. It was trained on over a million images belonging to 1000 classes comprising real-world objects such as animals, cars, and tables.

Since the histological images did not belong to any of the 1000 classes, the last layer of the VGG-16 architecture was not necessary in this experiment. It would have given the probabilities of which of the 1000 classes the images belonged to, none of which are true in this case. Also, the second to the last layer had 4096 values, which would be more informative in the current prediction. Training a new CNN model with a purely histopathological image dataset should further improve prediction accuracy but it would take considerable effort to collect millions of images and model training is expected to take several days to complete. A smaller image database would still work but larger datasets should give better classification accuracies.

With its current track record, CNN holds promise in augmenting the practice of pathology. Potential applications include pre-screening slides for malignancy and aiding the pathologist in searching for mitosis and counting different categories of cells in bone marrow smears, both tasks considered mundane and boring for a pathologist.

Jerome Cheng is a Clinical Assistant Professor of Pathology in the Division of Pathology, University of Michigan. His research interests include image analysis, natural language processing, data mining, and machine learning of medical datasets.

The Digital Pathology Congress is returning to London in December. The agenda includes a full track of presentations on computational pathology and AI. Take a look here. 

Tags: computer programming, deep learning, digital pathology, machine learning