Speaker
Description
Workshop Relevance: For Astrophysics and the Medical/Public Health Sciences, the opportunity for machine learning to complement rule-based image routing or alert systems is shared. In this work, we demonstrate how deep learning, random forests, and an ensemble of the two can aid and improve medical image (MRI) routing systems. This work has parallels to astronomical datasets such as Photometric LSST Astronomical Time-Series Classification Challenge (https://plasticc.org/) where the creation of an observation routing or alert system based on machine learning algorithms can prove effective. In particular, these systems may assist in collecting observations of similar transient phenomena and celestial objects across observatories and nights, allowing us to learn more about their behavior. We have created a system with such an objective for medical imaging, and present it to the Earth meets Sky community for discussion on how such a tool can assist astronomical surveys.
Background and Purpose: Increasingly complex MRI studies and variable series naming conventions reveal limitations of rule-based image routing, especially in health systems with multiple scanners/sites. Accurate methods to identify series based on image content would aid post-processing and PACS viewing. Recent deep/machine learning efforts classify 5-8 basic brain sequences. We present an ensemble model combining a convolutional neural network (CNN) and a random forest classifier (RFC) to differentiate 23 sequences and image orientation.
Materials and Methods: Series were grouped by descriptions into 23 sequences and 4 orientations. Dataset A, obtained from our institution, was divided into training (20,904 studies; 86,089 series; 235,674 images) and test sets (7,265 studies; 62,223 series; 3,658,450 images). Dataset B, obtained from a separate hospital, was used for out-of-domain external validation (1,252 studies; 2,150 series; 234,944 images). We developed an ensemble model combining a 2D CNN with a custom multi-task learning (MTL) architecture and RFC trained on DICOM metadata.
Results: Dataset A overall accuracy by RFC was 97%, CNN 97%, and ensemble 98%. Dataset B overall accuracy by RFC was 99%, CNN 98%, and ensemble 99%. Error analysis revealed different types of discrepancies: non-brain studies; incorrect ground-truth labels; and incorrect predictions, some with possible explanations.
Conclusion: The ensemble model for series identification accommodates the complexity of brain MRI studies in state-of-the-art clinical practice, performing slightly better than the CNN and RFC separately. Expanding on previous work demonstrating proof-of-concept, our approach is more comprehensive with more classes and orientation classification.
Future Work: Using a network trained on this data as a tool for fine-tuning on downstream tasks, scaling up to predict more attributes of medical images, and exploring the potential of an analogous system to be inserted in astronomical observation systems.
