How Deep Learning-Based Image Classification Techniques Are Taking Over Medical Imaging

Deep learning-based techniques are efficient for early and accurate diagnosis of disease, helping healthcare practitioners save many lives.

In the past few years, deep learning-based techniques have evolved and revolutionized many industries, including healthcare. In this article, we’ll discuss medical imaging and the evolution of deep learning-based techniques.

What is medical imaging and why it is important?

Medical imaging comprises different techniques to create visual representations of internal parts of the human body, like tissues or organs, to monitor their functioning, diagnose, and treat diseases. Medical imaging also adds to anatomy and physiology databases.

Medical imaging techniques include radiography, MRI, ultrasound, endoscopy, thermography, tomography, and so on. Different techniques provide tailored information related to specific areas of the human body.

There are many players manufacturing medical imaging devices, which include Siemens Healthineers, Hitachi, GE, Fujifilm, Samsung, and Toshiba. There is a huge global market for medical imaging devices, which is expected to grow to a whopping $48.6 billion by 2025.

High-quality images provided by different medical imaging techniques can improve the decision-making process and avoid unnecessary medical procedures. These techniques can diagnose many diseases and injuries like cancer, pneumonia, brain injuries, internal bleeding, and so on.

Issues with traditional machine learning techniques

The interpretation and understanding of medical images are limited because of different parameters, complexity, and requirement of core subject knowledge. In the past, people tried to use machine learning algorithms like logistic regression, decision trees, support vector machines, and so on, to understand medical images.

This approach has some limitations:

• These traditional machine learning algorithms rely heavily on carefully crafted features by subject matter experts, which is a demanding process
• Medical images vary among patients, and feature generation also differs among subject matter experts. This questions the reliability of this traditional approach
• Traditional machine learning algorithms process raw image data without taking hidden and subtle representations into account. This results in low performance compared to deep learning-based algorithms
• Traditionally, feature descriptors like HOG, SIFT, SURF, etc., were used to generate feature vectors, and classification algorithms used the generated feature vectors to classify images

In contrast, deep learning-based algorithms capture hidden and subtle representations and automatically process raw data and extract features without requiring manual interventions. These benefits over traditional approaches lead to their fast adaptation in medical imaging, as mentioned in the next section.

Evolution of deep learning-based image classification techniques

Evolution started from AlexNet, the first neural net to win the ILSVRC image classification competition back in 2012. After that, many architectures came that include VGG Net, Inception (GoogleNet), ResNet, etc. Recently, Vit-H/14 and FixEfficientNet-L2 are in first and second positions respectively on ImageNet leaderboard according to Top-1 accuracy.

The use of pre-trained models for other applications using the fine-tuning technique opened endless possibilities without the need for training models from scratch. However, pre-trained models on medical images are hard to find and are an area of improvement.

Get To Know Other Data Science Students

Mengqin (Cassie) Gong

Data Scientist at Whatsapp

Read Story

Diana Xie

Machine Learning Engineer at IQVIA

Read Story

Hastings Reeves

Business Intelligence Analyst at Velocity Global

Read Story

Application of deep learning in medical imaging

Here are some areas where deep learning is applied:

• Tumor detection. In 2019, Shen et al applied Deep Convolutional Neural Network (DCNN) based approach on a mammography database to detect breast cancer.
• Cardiac image analysis. A review paper by Litjens et al describes different deep learning-based approaches for cardiovascular analysis.
• Parkinson’s and Alzheimer’s detection. Paper by Oritz et al discusses isosurface-based features and Convolutional Neural Network (CNN) based approach for Parkinson’s disease detection. In 2016, Sarraf and Tofighi used DCNNs to classify Alzheimer’s disease using fMRI data. Jha and Kwon took the autoencoder based approach for Alzheimer’s disease detection.
• Diabetic Retinopathy. In 2016, Chandrakumar and Kathirvel used DCNNs to classify diabetic retinopathy. Paper by Gulshan et al discussed the use of DCNNs for diabetic retinopathy detection from retinal fundus images
• Gastrointestinal Diseases Detection. In 2018, Urban et al used DCNNs to identify and localize polyps in real-time. Paper by Leenhard et al described the use of DCNNs for the detection of GI angioectasia while performing small-bowel capsule endoscopy.

Current challenges for deep learning in medical imaging

Although the deep learning based-approach is suitable for medical imaging, there are some challenges we need to address:

1. Dataset availability. Medical imaging data is very limited, which is a big challenge for deep learning-based techniques. Developing and labeling datasets require subject matter experts, and it is a very time-consuming and demanding process—especially for rare illnesses. In addition, most of the medical datasets are imbalanced, which poses another problem.
2. Interpretability of models. Although deep learning-based models provide good accuracy for medical imaging, their uninterpretable nature is a big question behind whether correct features and representations are learned or not.
3. Data standardization. As mentioned above, medical datasets are limited, and often these datasets are formed by gathering data from different institutes that use diverse hardware. This increases variability in the dataset.
4. Data privacy. As medical data has personal information of patients, it poses high privacy restrictions and reduces the availability of data for the use of data science researchers and organizations.
5. Biased data. Biased data can reduce model accuracy and lead to risky predictions in the production environment. For example, if a breast cancer detection model is only trained with x-ray data from white women, then this model can be biased towards predictions on Black women’s x-rays when deployed in production.
6. Risk of wrong prediction. If a model either wrongly identifies a tumor, or does not detect one, the consequences are very heavy for the patients. For example, if a breast cancer classification model fails to predict the existence of breast cancer in a woman’s x-ray, then it can be fatal. We might know cancer exists when it’s too late because the model predicted it wrong initially when it was curable.

Apart from these challenges, deep learning-based techniques are improving and, in some cases, increasing performance beyond subject experts. Moreover, Data Scientists and ML Engineers are working  collaboratively round the clock  to crack these challenges. Currently, it is positioned as a great assistant to medical experts, rather than a replacement.

Check out Kaggle’s medical datasets and competitions to explore applications of machine learning in medical imaging.

Since you’re here…
Thinking about a career in data science? Enroll in our Data Science Bootcamp, and we’ll get you hired in 6 months. If you’re just getting started, take a peek at our foundational Data Science Course, and don’t forget to peep our student reviews. The data’s on our side.