App turns smartphone cameras into diagnostic tool

Using cartoonlike images, test looks for vision problems in young children

Chinese scientists have developed a smartphone-based deep learning system for the early detection of visual impairments in young children, which only requires them to watch cartoonlike images for three minutes to be tested.

A report on the system was recently published in the international science journal Nature Medicine. The research was conducted by a team led by Lin Haotian, a professor at the Zhongshan Ophthalmic Center at Sun Yat-sen University.

Called Apollo Infant Sight, the system is designed to elicit a steady gaze through the use of cartoonlike video imagery to capture children's attention. Cameras record their gaze and facial features for further analysis by deep-learning models, Lin said.

Capable of identifying 16 disorders, including congenital cataracts, congenital ptosis and congenital glaucoma, the system has an average accuracy of around 85 percent.

Infancy is a critical period for visual development, during which early detection and prompt treatment of emerging problems can prevent irreversible sight loss.

However, detection has proved difficult in the past as young children are easily distracted during standard vision tests, Lin said.

Age-appropriate tests such as acuity cards are commonly used to observe the reactions of infants and toddlers to visual stimuli.

However, they require trained practitioners to administer, which hinders wider adoption. This is especially true in low-income and middle-income countries, which have the highest prevalence of visual impairment but lack medical resources.

In the study, more than 25 million video frames of 3,652 children were collected and entered into a deep-learning-based back end deployed in the Tianhe-2 supercomputing center in Guangzhou, Guangdong province.

Two facial detection and key point localization models were trained on publicly available data sets from an open-source library. The team developed neural networks using images sampled from raw videos collected from the Zhongshan Ophthalmic Center.

To test the stability of the system in more complex settings, researchers performed adjustments to a data set to simulate the diversity of the data quality of different smartphone cameras.

The system was evaluated at five different clinics and validated under varying conditions in real-world settings.

During research, the mobile phone app guided users, including healthcare professionals, volunteers, parents and caregivers, interactively to familiarize them with the system and complete standardized preparations, including choosing a suitable test setting.

Test results were displayed on the app to alert users of the need to refer children found to be at high risk of visual impairment to experienced pediatric ophthalmologists.

Even when dealing with diseases not covered by the training set, the system showed effective discriminative capabilities, indicating its extendibility to diagnosing other conditions, and it has the potential to be used by healthcare professionals, parents and caregivers, Lin said.

Future work is needed to evaluate the merits of the system in detecting visual impairment caused by a broader range of diseases, such as cerebral visual impairment, and in reducing extraocular morbidity associated with systemic diseases in the larger population, such as those resulting from cardiovascular complications linked to Marfan syndrome.

To protect privacy, techniques such as lightweight model backbones and model pruning could be applied to deploy the deep learning system in individual smartphones, which would not require additional computing resources.

Digital fingerprint technology such as blockchain could also be applied to monitor data usage and mitigate abuse.

The research team developed real-time, three-dimensional facial reconstruction technology to erase biometric attributes while retaining gaze patterns and eye movements, which can be used in the future to protect the privacy of those being tested, he said.

The team is in discussions with online public platforms to upload the application so that people in Guangdong are easily able to use it. Currently, the app is available as a WeChat mini program from the Zhongshan Ophthalmic Center.