Artificial Intelligence and Diagnosis based on Multiple Data Sources

This project is a large interdisciplinary partnership of Doctors, Public Health Researchers, Data Scientists, Physicists and Engineers. It combines the data sets represented by several sources, including images of chest X-rays, audio recordings of coughs and set sentences, and Natural Language Processing applied to clinician data of patient interviews, and possible from wearable diagnostic devices. The Artificial Intelligence (AI) solution is a weakly supervised Deep Learning network. Its novel in the range of data sources which are unified and then presented coherently to a common AI engine and also in the interaction between the interdisciplinary team in developing the system. Ultimately one may have a tool that can support the diagnosis of a broader range of people with lung pathologies, even over the telephone (COVID-19, pneumonia, TB etc). Based on the wearables, the machine learning algorithm would also be able to diagnose if the patient is crashing or stable.
The Institutions participating in this research project are the Perinatal HIV Research Unit (PHRU), Right To Care (RTC), the University of Johannesburg (UJ), the Brookhaven National Laboratory in the USA (BNL), the Technical University of Tshwane (TUT) and former students from the African School of Physics. This is a project where Artificial Intelligence is used to process a variety of data sources specific to each patient in a context of assisted diagnosis for lung pathologies, including COVID-19, pneumonia and Tuberculosis. The several data sources include images of chest X-rays, audio recordings of coughs and set sentences, and Natural Language Processing applied to clinician data of patient interviews, and possibly also ultimately from wearable diagnostics. This combined data set is then processed by an Artificial Intelligence (AI) algorithm. The project is novel in the range of data sources which are unified and then presented coherently to a common AI engine and also in the interaction between the interdisciplinary team in developing the system. The hypothesis is that the X-ray data, the cough and the voice data and the clinician data are linked, and if they change in time in successive presentations, then this trajectory is also longitudinally linked. The goal is a set of tools that can support the diagnosis of a broader range of people with lung pathologies. The longitudinal component of the study could identify changes, and perhaps allow a timeous medical intervention. The groundwork for the project began some time ago, with the collection and archiving of the data. In fact, there is already a significant database of all three data types (X-ray images, Audio bytes, Clinician data). The clinician data arises from digitised scans of multiple documents from previous research studies and is being supplemented continuously. The digitised records are subjected to optical character recognition, from whence Natural Language Processing (NLP) can be applied. The AI approach will be designed to identify and extract morbidity and mortality-relevant keywords and other relevant data from the patient medical files. These data will be aligned and merged with the X-Ray and audio byte data. This data also has a longitudinal component as the same patients revisit the clinic and are re-assessed. Ultimately one may have a tool that can support the diagnosis of a broader range of people with lung pathologies, as they present and are assessed in detail, or even simply from a distance, over the telephone. This will allow for better patient management and care. In this project we want to develop and pilot the system. The project has dual research opportunities in both Medical and Artificial Intelligence spaces. Both are to be addressed here.