Computer-aided Analysis of Fundus Photographs

There are many advantages of using digital image analysis to quantify the extent of retinal pathology in vascular diseases, diabetic retinopathy (DR), age-related maculopathy and other conditions. The key benefits include immediate viewing, image management systems that allow disease progression to be monitored by reviewing sequential images and patient education.

There are many advantages of using digital image analysis to quantify the extent of retinal pathology in vascular diseases, diabetic retinopathy (DR), age-related maculopathy and other conditions. The key benefits include immediate viewing, image management systems that allow disease progression to be monitored by reviewing sequential images and patient education.
The resolution of the newest generation of ocular digital photographic systems (at 4.000 pixels and higher) is similar to that of film. Digital images can be displayed on a video screen as soon as they are obtained, making it possible to detect and correct any error in the photographic process immediately.
If the patient blinks, the focus is not perfect, or the flash is too intense, additional images can be obtained at minimal expense. Additionally, the wide use of digital systems makes it possible to use telecommunication technology. Fundus and angiographic images are already being sent to remote digital reading centres for clinical evaluation or research purposes.

Colour Fundus Images

Colour fundus imaging is the most frequently used imaging modality because it is non-invasive, well accepted by patients and, above all, because it allows the visible state of the retina to be recorded at a particular point in time, allowing documentation and a deeper and extended analysis.
After recording a particular retina visible state over time, a basic and widely used technique is to evaluate the temporal sequence to identify visible changes in the retina. Retinal lesions such as haemorrhages and exudates and their progression over time are important indicators of retinal disease activity.

Fundus Camera Digital Imaging

Morphology of Lesions

Microaneurysm Location Identification

The location of each microaneurysm may be identified by automated analysis, allowing for counting newly formed and disappearing microaneurysms and the calculation of microaneurysm turnover.^1,2

Hard Exudates

Automated analysis allows identification of changes and progression.³ It is an indirect measure of leakage and alteration of the blood–retinal barrier.

Vessel Diameter Irregularity

Irregularities in the regular shape of the vascular walls (arterioles and venules) may be detected by automated analysis of digital images, without being influenced by short-term changes in heartbeat, etc.

Vascular Occlusion

The relationship between the vascular tree density and sparsity is clearly a potential method to detect vascular occlusion and may be a useful indication for the use of anti-vascular endothelial growth factor medication.^4,5

Location of Lesions

Involvement of the fovea resulting in central visual acuity loss is well detected by identifying lesions within the central 1,000 μm diameter circle of the posterior pole.

Retinal Lesion Changes Over Time

This article focuses on automated computer-aided analysis of fundus photographs to identify retinal changes occurring over time, especially microaneurysm turnover.

Image Co-registration

A mandatory step for automatically comparing images is their co-registration, one against the other. That is, one of the images needs to be projected to the image space of the other, which acts as a reference image. Following this procedure, both share a common reference making it possible to establish a direct pixel-to-pixel correspondence.
To achieve the required image co-registration, it is necessary to identify eye fundus natural landmarks, intrinsic fiducial marks and compute the transformation matrix that, applied to one image, will project it to the image space of the reference image.
Two major steps are involved in the above concept. One relates to the identification and classification of the fiducial markers, while the other relates to linking similar fiducial markers between any two images to co-register.
A natural source for fiducial markers is the retinal vascular network, an imprint for each human eye. Vessel characteristics, bifurcations and crossovers make it possible to establish links between any two images from the same eye. After finding the true links for several fiducial markers, the respective transformation matrix can be computed. These two steps are implemented using a proprietary technique while the vascular network is segmented into contourlets, following an initial approach of using differential geometry.¹

Automated Monitoring of Diabetic Retinopathy Progression – Microaneurysm Turnover

It is of fundamental importance to monitor disease progression in specific patients and identify if they shows signs of rapid progression and to which phenotype of progression they belong. Some patients need special attention and timely intervention to avoid development of DR complications, macular oedema or proliferative DR.
The main changes that occur in non-proliferative diabetic retinopathy (NPDR) and need to be monitored are microaneurysm dynamics (their formation and disappearance), vascular leakage with subsequent oedema and hard exudates formation and capillary closure.
Visual function loss occurs characteristically late in DR because the eye has a large functional reserve of vision and DR initially affects the inner layers of the retina away from the photoreceptors. Therefore, structural changes are detected in DR earlier than functional changes. Evidence of structural changes need to be the focus to follow progression in the earliest stages of DR.
One of the best candidates for non-invasive imaging of the eye fundus is fundus digital photography because retinal cameras are widely available and the data obtained may be supported and analysed by computer-assisted procedures.
To identify progression it is essential to collect sequential series of images and these images must be compared. The need for co-registration of these sequences of images is, therefore, of great relevance. By applying novel image co-registration procedures and automated comparative analysis software it is now possible to perform reliable sequential comparisons of fundus digital photography images.
RetmarkerDR (Critical Health, Portugal) is software that is currently available and is able to automatically detect changes occurring in eye fundus digital images. This software compares successive visits to the reference image, in each eye, based on co-registration and co-localisation of the changes (Figures 1 and 2).
Using fundus photography, microaneurysms and small haemorrhages are the initial changes detected in the diabetic retina. Retinal microaneurysms may be counted and this technique has previously been suggested as an appropriate marker of retinopathy progression.^6,7 Retinal microaneurysms are important lesions of diabetic retinopathy and even one or two microaneurysms in an eye should not be regarded as insignificant.⁸ When examining 1,809 patients in the United Kingdom Proliferative Diabetes Study (UKPDS) cohort that had either no retinopathy or microaneurysms only at study entry, the number of microaneurysms were found to have a high predictive value for worsening retinopathy at three, six, nine and 12 years after entry into the study.⁸ Similar findings were presented by Klein et al., who looked at the relationship between retinal microaneurysms and the progression of diabetic retinopathy over a four-year period.⁹ In this study the number of microaneurysms at the baseline examination was positively associated with significant progression of retinopathy. More recently, Sjølie et al. showed that microaneurysm counts were predictive of an increased risk of retinopathy progression.¹⁰
Our studies have shown that it is not the absolute total number of microaneurysms at a certain point in time that may provide the best indication of retinopathy progression, but the rate of microaneurysm turnover in successive visits during a one- or two-year period. We have demonstrated that it is possible to use microaneurysm turnover computed from non-invasive colour fundus photographs as a biomarker to identify patients at risk of progression to clinically significant macular oedema (CSME).² A microaneurysm formation rate of at least two microaneurysms per eye in eyes with mild NPDR and diabetes type 2 also appears to identify patients at risk for progression to CSME. In one recent study, with a 10-year follow-up of 113 eyes/patients, the percentage of false negatives (eyes that developed CSME with a low microaneurysm formation rate) was 29.4 % (5/17) and the percentage of false positives (eyes that did not develop CSME with a high microaneurysm formation rate) was 8.3 % (8/96), resulting in a sensitivity in predicting CSME development of 70.6 %. The high negative predictive value for CSME (94.6 %, 88/93) indicates that a low microaneurysm turnover, i.e. fewer than two microaneurysms per year, identifies particularly well the eyes that are not expected to progress to CSME within a 10-year period.
Differences between successive visits using microaneurysm counts are less reliable than microaneurysm formation rates, which take into account newly formed microaneurysms and give more accurate information on activity of the retinopathy.¹¹ Furthermore, we have also found much better agreement between graders when determining microaneurysm turnover than in microaneurysm counting alone.¹
Recently, Sharp et al.¹² found that the microaneurysm turnover varied widely among eyes of the same retinopathy level; this is consistent with our findings. Microaneurysm turnover has been shown in our studies to vary among eyes classified with the same retinopathy level.¹³ Particularly relevant and of major interest is the finding that patients who have higher microaneurysm turnover values go on to develop CSME and show more rapid retinopathy progression, particularly in association with poor metabolic control demonstrated by higher haemoglobin A1c values. Therefore, microaneurysm turnover appears to be a distinctive characteristic that indicates activity of disease and rapid progression in eyes with apparently similar retinopathy level.
A more recent study, using automated analysis of digital fundus photographs with the RetmarkerDR software, confirms our initial observations and demonstrates that automated analysis of fundus digital images is extremely promising in determining DR progression. Furthermore, it is an easy to use non-invasive tool to predict development of CSME. In the Caldiret study, co-ordinated by Munich University, a group of 290 eyes were followed by fundus photography for five years. It was possible to compare the 49 eyes that did develop CSME over the study period with the 241 eyes that did not develop CSME in the same five-year period.¹⁴ In this study, an increased microaneurysm formation rate was confirmed to be clearly associated with the development of CSME. Values of microaneurysm formation rate greater than two per year in this early stage of retinopathy were present in 71.4 % of the eyes that developed CSME. In clear contrast, the eyes that did not develop CSME during the period of the study showed a microaneurysm formation rate of less than two per year in 71.8 % of the cases.

Identification of Risk Profiles Using Non-invasive Procedures

To evaluate progression of NPDR it is necessary to be able to closely follow any significant changes in the three major features of NPDR: microaneurysms, alteration of the blood–retinal barrier and capillary closure.
Microaneurysms are a sign of vascular disease and their formation and disappearance, i.e. their turnover rate, appear to be the best indicators of disease activity. In the initial stages of NPDR a microaneurysm formation rate of two or more per year is a predictor of progression and the development of macular oedema.¹⁵
Capillary closure is a relevant step in the retinal vascular disease process that leads to progressive ischaemia and formation of preferential channels inducing remodelling of the retinal circulation. The presence of ischaemia is a necessary condition in the development of proliferative retinopathy. The development of capillary closure may be identified using colour fundus photography by determining the rate of microaneurysm disappearance or by changes in the same eye in the branching pattern of the arterioles and their density in the macula.
Alterations of the blood–retinal barrier cannot be identified by colour fundus photography but may be identified using another non-invasive method, optical coherence tomography (OCT).¹⁶ Measurements of retinal thickness and changes in the retinal extracellular space resulting from focal alterations of the blood–retinal barrier occurring in the initial stages of NPDR can be performed and show the degree of breakdown of the blood–retinal barrier and its change over time. Work in progress in another group is also focusing on the ganglion cell layer using OCT to determine ganglion cell loss as an indicator of ischaemia.¹⁷
Incorporating information about progression risk can be difficult in clinical practice but is an objective that must be pursued if we want to improve management of DR and prevent the development of the major complications of DR: macular oedema and proliferative retinopathy.
Creation of a progression profile starts with establishing an adequate baseline. The first step is clinical assessment and documentation of patients with diabetes, identifying the presence of NPDR in its initial stage (equivalent to Wisconsin classification <43). This baseline should include digital fundus photography with at least two fields, one centred on the fovea and the other on the optic disk and OCT with central retinal thickness measurements and an optical density baseline and best corrected visual acuity obtained, preferably using Early Treatment Diabetic Retinopathy Study (ETDRS) procedures.
Subsequent testing should include the same procedures after six or 12 months and repeated again 24 months after the collection of the baseline. Automated computer-aided analysis of the fundus photographs will give the rate of progression of retinal vascular disease. The rate of formation of new microaneurysms indicates activity of retinal disease with the rate of disappearance of microaneurysms indicating the occurrence of capillary closure. Changes in OCT will monitor alteration of the blood–retinal barrier and how the retinal vessels are responding to changes of metabolic control in an individual eye. The next step is to develop a risk profile based on the patient’s ocular phenotype of progression and non-ocular risk factors.
Determination of the activity of the retinal disease and individual risk profiles using non-invasive procedures are expected to contribute decisively to better personalised management of diabetic retinopathy and prevent vision-threatening complications – macular oedema and proliferative retinopathy.

Automated Computer-aided Analysis of Fundus Digital Photographs in Diabetic Retinopathy Screening

The development of systematic programmes of screening for retinopathy has been identified as an urgent healthcare need. Studies have indicated that the severity of vision loss due to diabetes is largely due to lack of screening.¹⁸
We have developed and evaluated a novel two-step approach that automatically screens colour fundus photographs in patients with the use of sequential examinations from the same patient to analyse the evolution of the disease in that patient.¹⁹ The automated grading system, RetmarkerSR consists of software earmarking microaneurysms and ‘red-dot-like’ vascular lesions. The system includes a co-registration algorithm that allows comparisons between different screenings for the same eye in the same retinal location. In a first-step single analysis, the system generates one of two possible outputs – ‘disease’ or ‘no disease’. The ‘disease’ category includes images where vascular lesions are found in the central macula that correspond to level 35 and above of the ETDRS scale, including maculopathy, advanced non-proliferative retinopathy and proliferative retinopathy. In the one-step analysis, the algorithm detects the presence of red-dot-like lesions in fields 1 and 2 (field 1 is centred on the optic disc and field 2 is centred on the fovea). The algorithm combines this initial analysis with a second analysis that compares two different and consecutive examinations for the same patient from two successive screenings with an interval of approximately one year. The images from the field centred on the macula are co-registered to complete a difference analysis that will indicate disease activity in the central 3,000 μm diameter circle of the macula. The results show a clear improvement over currently available fully automated screening algorithms, with a sensitivity of 95.8 % and a specificity of 63.2 %. RetmarkerSR was shown to identify urgent cases for referral and will therefore allow the burden of manual grading to be reduced. This two-step analysis shows a clear improvement in specificity over other available automated systems.^20,21 The integration of this technique into a yearly screening programme is expected to result in a progressive decrease in the burden of manual grading by safely decreasing the number of false-positive results to be manually graded, with economic advantages, making DR screening more feasible. ■