UP2SMART’S SOLUTIONS FOR EYE HEALTHCARE
Diabetic Retinopathy (DR) is a neurovascular complication of Type 1&2 Diabetic patients that leads to complete vision loss. DR is the major cause of preventable blindness in working-age adults. Globally, DR is the fifth cause of blindness and visual impairment affecting 3-4% of Europeans. Moreover, DR affects 30% of diabetic patients and it is estimated that 280 million people are blind by diabetes in the world.
When detected on time, the risk of DR can be reduced by 95% but the disease is asymptomatic in its early stages and the risk of developing DR varies among patients. Diabetic Ophthalmology Associations recommend regular screening, while this can be done through telemedicine, most patients are still screened at ophthalmology units.
There are no standardized screening programs across Europe and the implemented solutions at the national level are notably expensive, time-consuming, and not fully adequate for the patient’s needs. Besides, existing healthcare systems are inefficient in detecting DR due to: 1) A high number of diabetic patients causing an excess of patients per available mydriatic camera; 2) Most patients are being screened at the ophthalmology units, even though there is the option of telemedicine with non-mydriatic fundus cameras; 3) This screening is more challenging outside urban areas (29% of Europeans), where ophthalmologist units containing non-mydriatic cameras are located.
UP2SMART platform for eye healthcare
UP2SMART developed a platform for early detection and diagnosis of DR combining artificial intelligence (AI) models. It includes two AI-based systems: MIRA and RETIPROGRAM. Mira is a fully automated eye fundus image reading system while RETIPROGRAM is a diagnostic assistance tool that uses AI techniques to model electronic health record data (EHR) of the population to predict the risk of developing DR.
UP2SMART’ platform can supply a real-time response that allows the health centre physician to make a decision for the subsequent steps in the treatment of a patient if it is required. Either the next check-up or diagnosis by a specialist in the ophthalmologist's office in the hospital in the event that the platform had found something possibly abnormal to be assessed by the specialist.
UP2SMART’ platform can improve the efficiency of DR screening, the productivity of healthcare systems and patient satisfaction. It mitigates the limitations of DR screening considering both the economic costs and patient outcomes, as follows:
- Help optimize scheduling indication, priority, revision and prioritization of diagnostic procedures according to pathology and clinical symptoms.
- Help non-eye specialised healthcare professionals (primary care doctors) in stratifying DR patients.
- Provide a personalized screening that will improve patient outcomes and satisfaction.
- Improve interaction between main hospitals and primary care centres to provide better and timely support and decrease the burden on families.
- Help reduce the number of patients reaching the later stages of DR and decrease treatment costs.
Diabetic Retinopathy (DR) is a chronic disease and one of the main causes of blindness and visual impairment for diabetic patients. Diabetic retinopathy is one of the most common morbidities associated with diabetes mellitus (DM). Normally, the appropriate control of the disease requires the implementation of expensive screening and following up programs. Thus, RETIPROGRAM and MIRA systems aim at helping doctors to identify and follow up patients with DR using artificial intelligence (AI) based tools extensively studied in a multidisciplinary research that has included ophthalmologist experts in retina.
RETIPROGRAM: A clinical decision support system for Diabetic Retinopathy early detection
This program is a Clinical Decision Support System that helps to discriminate between diabetic people that may develop Diabetic Retinopathy and others who do not have any risk.
The computer program is focused on people with Type-2 diabetes. It used 9 input risk factors that can be obtained from the Electronic Health Record of the patient, or introduced by a physician. Then, the system uses a set of fuzzy classification rules and a procedure of aggregation to determine the degree of risk of developing Diabetic Retinopathy.
The users of this system are doctors (family physicians or ophthalmologists), who receive a prediction of DR risk together with a degree of confidence for the result. Given that information, the system is able to calculate the most appropriate date for the next control visit of each patient.
It is not for personal use of the patients as some of the data required needs to be validated by the expertise of the doctor. RETIPROGRAM is a decision support tool for helping doctors to determine the best screening periodicity for each person in order to assure appropriate eye health care. The system also helps to guarantee that human, material and economic resources are more efficiently employed.
RETIPROGRAM, has been constructed with Artificial Intelligence techniques and is the result of an extensive multidisciplinary research (since 2007) including ophthalmologists’ experts in retina. Some of the scientific results supporting the system are summarized in the list of publications shown below.
- A clinical decision support system for diabetic retinopathy screening: creating a clinical support application. Pedro Romero-Aroca, Aida Valls, Antonio Moreno, Ramon Sagarra-Alamo, Josep Basora-Gallisa, Emran Saleh, Marc Baget-Bernaldiz, Domenec Puig. Telemedicine and e-Health 25 (1), 31-40. 2019. https://www.liebertpub.com/doi/full/10.1089/tmj.2017.0282.
- Learning ensemble classifiers for diabetic retinopathy assessment, Artificial. Emran Saleh, Jerzy Błaszczyński, Antonio Moreno, Aida Valls, Pedro Romero-Aroca, Sofia de la Riva-Fernández, Roman Słowiński, Intelligence in Medicine, Volume 85, 2018, Pages 50-63. 2018. http://www.sciencedirect.com/science/article/pii/S0933365717300593.
- Glomerular filtration rate and/or ratio of urine albumin to creatinine as markers for diabetic retinopathy: a ten-year follow-up study. Pedro Romero-Aroca, Marc Baget-Bernaldiz, Raul Navarro-Gil, Antonio Moreno-Ribas, Aida Valls-Mateu, Ramon Sagarra-Alamo, Xavier Mundet-Tuduri. Journal of Diabetes Research 2018. https://www.hindawi.com/journals/jdr/2018/5637130/.
- Changes observed in diabetic retinopathy: eight-year follow-up of a Spanish population. Pedro Romero-Aroca, Sofía De La Riva-Fernández , Aida Valls-Mateu Ramon Sagarra-Alamo , Antonio Moreno-Ribas , Nuria Soler. The British Journal of Ophthalmology ;100:1366–1371. 2016. https://pubmed.ncbi.nlm.nih.gov/26769672/.
- Cost of diabetic retinopathy and macular oedema in a population. An eight year follow up. Pedro Romero-Aroca, Sofía De La Riva-Fernández , Aida Valls-Mateu Ramon Sagarra-Alamo , Antonio Moreno-Ribas , Nuria Soler, Domenec Puig. BMC Ophthalmology 16:136. 2016. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4973531/.
- PhD thesis: Screening study and development of a clinical decision support system in diabetic retinopathy. 2017. (document available in spanish) https://www.tdx.cat/handle/10803/450863.
Screenshot of Retiprogram web interface (www.neverblind.ai)
The RETIPROGRAM system requires 9 input values in order the system is able to perform the patient’s risk of developing DR. These input values are described below:
- Age: current patient’s age in years
- Gender: man or woman
- EVOLUTION: years since the beginning of the diabetes
- TTM: type of treatment: diet, oral antidiabetics or insulin.
- HbA1c: number representing the hemoglobina glicosilada.
- CKDEPI: Estimated glomerular filtration rate
- MA: Microalbuminúria.
- BMI: Body Mass Index
- HTAR: Hypertension control : good control or bad control.
Once the system performs the analysis it provides as output the risk (positive or negative) of developing DR with the respective certainty of the prediction. The next follow up visit is also recommended for the patient according to the risk predicted from the patient’s input data. If the system is not able to make a prediction it indicates unknown.
The system has been trained using a private dataset collected, reviewed and labelled by ophthalmologists experts in retina who have worked closely with the team working in the artificial intelligence algorithms. The current system was built from a sample of 3337 patients within a population of 15811 patients screened from 2007. The sample dataset was divided into a training set of 2234 patients and a testing set of 1103 patients. The RETIPROGRAM system performance is shown below. More technical details can be found in the scientific support publications.
- Accuracy > 81 %
- Specificity > 81 %
- Sensitivity > 80 %
MIRA System: Classifier of eye fundus images in to detect the degree of Diabetic Retinopathy
Preventive screening of the eye fundus is a usual technique to detect Diabetic Retinopathy disease (DR). Automatic tools for image processing and classification can facilitate the hard work of manual labelling of this kind of images, which requires high expertise and a lot of time from expert doctors.
An eye with DR may present some of the following lesions: microaneurysms, exudates, hemorrhages. The number, location and level permits to classify the disease in different stages. The computer System MIRA receives an eye fundus image and classifies the image into 4 categories: No DR, Mild DR, Moderate DR and Severe DR. In addition, in the preliminary stage permits to filter non-valid images, which can be images that do not correspond to an eye, or images with poor quality that should not be analysed. This tool avoids the classification in a DR level of images that are wrong.
The tool has been constructed using Deep Learning techniques for training 3 different models using a data set of labelled images. Ophthalmologists experts in retina have labelled and validated the images to assure the best performance in the system.
This system is not for use of patients, it is a support tool for doctors, who can check if the prediction given by MIRA really corresponds to what is observed in the image. In that sense, MIRA can be especially useful to avoid False Negatives, because the automatic classification system is able to detect very small lesions that are hardly visible by human inspection.
The MIRA system is the result of an extensive multidisciplinary research in collaboration with ophthalmologists experts in retina. Some of the scientific results supporting the system are summarized in the list of publications shown below.
- A deep learning interpretable classifier for diabetic retinopathy disease grading. J de La Torre, A Valls, D Puig. Neurocomputing 396, 465-476. 2020 https://www.sciencedirect.com/science/article/abs/pii/S0925231219304539
- Validation of a Deep Learning Algorithm for Diabetic Retinopathy. Pedro Romero-Aroca, Raquel Verges-Puig, Jordi de la Torre, Aida Valls, Naiara Relano-Barambio, Domenec Puig, Marc Baget-Bernaldiz. Telemedicine and e-Health. 2019. https://www.liebertpub.com/doi/abs/10.1089/tmj.2019.0137
- Weighted kappa loss function for multi-class classification of ordinal data in deep learning. J de La Torre, D Puig, A Valls. Pattern Recognition Letters 105, 144-154. 2018. https://www.sciencedirect.com/science/article/abs/pii/S0167865517301666
- PhD thesis: Diabetic Retinopathy Classification and Interpretation using Deep Learning Techniques. 2019. https://www.tdx.cat/handle/10803/667077#page=1
- Improving the Performance of Diabetic Retinopathy Computer-Aided Diagnosis Systems Using an Ensemble of Texture Analysis Methods. S. Ragab, M. Abdel-Nasser, A. Moreno, D. Puig. Recent Advances in Artificial Intelligence Research and Development, Vol. 300, p. 16, IOS Press, 2017. http://ebooks.iospress.nl/volumearticle/47721
In this section it is described how to use the MIRA system to automatically analyse fundus images of eyes in order to detect if a person has Diabetic Retinopathy (DR) or not. In case the system identifies the analysed eye fundus image has DR, the system classifies the level of DR among Mild DR, Moderate DR and Severe DR.
The procedure that must be followed by a system’s user in order to analyse at least one fundus image is described below.
- The user must provide one or multiple fundus images with a minimum resolution of 640 pixels by 640 pixels. If the user provides images with a larger resolution, these will be automatically resized by the system. The accepted formats used by the system include: JPG, JPEG and PNG.
- Once the user has executed the graphic interface of the system and uploaded the images to be analysed, a visualization tool will show small views of each uploaded image on the left side. Later, the user must click on the first image that must be analysed.
- Once the eye image to be analysed has been chosen, the system initially determines if the image provided is a fundus image or not.
- Later, if the system identifies that the provided image is a fundus eye then the system performs a gradability test in which it is determined if the image has enough quality to be processed (correct illumination, focus, RGB intensity levels, blurriness).
- The system shows the gradability score to the user and displays a warning message if the gradability score is less than 50%. If the gradability score is more than or equal to 50%, the system performs the next step. The users have the option to continue to the next stage if the gradability score is above 25% and less than 50%. In the latest case, the reliability of the results could be degraded.
- In the last stage, if the quality of the image is good enough to be analysed by the developed deep learning model then the system automatically performs the diabetic retinopathy grading.
- Finally, in the last stage the system displays on a right-side panel the final results that indicate the DR grading for the current image analysed. A percentage with the probability is also displayed in order to indicate the reliability of the presented results.
Screenshot of MIRA system web interface for a negative DR patient (www.neverblind.ai)
Screenshot of MIRA system web interface for a positive DR patient (www.neverblind.ai)
The system has been trained using thousands of retinographies carefully reviewed and labelled by ophthalmologists experts in retina who have worked closely with the team working in the artificial intelligence algorithms. Our private dataset has been used for training and testing purposes for checking the validity of our trained models. Our dataset consists of 40862 retinograhies. The dataset was divided into a training set of 35862 retinographies and a testing set of 5000 retinographies. More technical details can be found in the scientific support publications.
For the identification of diabetic retinopathy from an eye fundus image, the system has the following performance using the testing set:
- Accuracy = 97 %
- Sensitivity = 88 %
- Specificity = 98 %