Dementia risk prediction in the population: are screening models accurate?

/Home/Alzheimer's Risk Factors/Alzheimer's Risk Factors - Frailty and Age

Early identification of individuals at risk of dementia will become crucial when effective preventative strategies for this condition are developed. Various dementia prediction models have been proposed, including clinic-based criteria for mild cognitive impairment, and more-broadly constructed algorithms, which synthesize information from known dementia risk factors, such as poor cognition and health. Knowledge of the predictive accuracy of such models will be important if they are to be used in daily clinical practice or to screen the entire older population (individuals aged ≥65 years). This article presents an overview of recent progress in the development of dementia prediction models for use in population screening. In total, 25 articles relating to dementia risk screening met our inclusion criteria for review. Our evaluation of the predictive accuracy of each model shows that most are poor at discriminating at-risk individuals from not-at-risk cases. The best models incorporate diverse sources of information across multiple risk factors. Typically, poor accuracy is associated with single-factor models, long follow-up intervals and the outcome measure of all-cause dementia. A parsimonious and cost-effective consensus model needs to be developed that accurately identifies individuals with a high risk of future dementia.

Stephan BC, Kurth T, Matthews FE, Brayne C, Dufouil C. Dementia risk prediction in the population: are screening models accurate? Nat Rev Neurol 2010;6:318–326.

Target Audience

This activity is intended for primary care clinicians, geriatricians, neurologists, psychiatrists, and other specialists who care for older patients.

Goal

The goal of this activity is to review prognostic models for assessing dementia risk, their characteristics, and predictive utility.

Learning Objectives

Upon completion of this activity, participants will be able to:

1. Identify the criteria for mild cognitive impairment (MCI) and the role of MCI in predicting dementia
2. Assess use of prognostic models to predict dementia and their utility in supplementing MCI
3. Describe the utility of multifactor models for predicting dementia


Introduction

The rise in the incidence of dementia with the change in the global age demographic is a source of major public health concern, as the disability associated with this condition, particularly in the later stages of disease, leads to high personal, social and economic costs. Given the promise of future preventive strategies to limit the expected increase in chronic neurodegenerative diseases, early and accurate identification of individuals who are at a high risk of developing dementia is regarded as a research priority. Numerous prognostic models that incorporate known dementia risk factors have been proposed to achieve this goal. Knowledge of the predictive utility of such methods is important if they are to be included in daily clinical practice or to screen the entire older population (individuals aged ≥65 years).

In this Review, we describe various dementia risk models that have been tested in population-based samples. The methodology that was used to select the articles for review is outlined in Box 1 . By using estimates of sensitivity and specificity, or the area under the receiver operating curve (AUC; Box 2 ), we assess the ability of each model to correctly classify at-risk individuals (that is, people who have developed dementia at follow-up) and not-at-risk individuals (that is, people who have not developed dementia at follow-up). A perfect prediction model has sensitivity and specificity values of 100%; however, in reality, such figures are rarely observed unless the screen is the diagnosis (for example, ultrasound for abdominal aneurysm). Generally, for population-based screening programs, high estimates (>90%) of both sensitivity and specificity are required to ensure that misclassification rates are low, although such threshold values can need adjustment with changes in disease prevalence. In this Review, sensitivity and specificity estimates of <80%, 80-89% and ≥90% are considered to represent measures of low, moderate and high accuracy, respectively. AUC values of 0.9-1, 0.8-0.9 and <0.8 are considered to reflect excellent models, good models and models of questionable utility, respectively. Following examination of the various dementia risk models, we discuss the limitations of current modeling approaches (including analytical considerations) and describe possible directions for future research.


___
Box 1. Detailed Article Selection Criteria

Articles were selected for review if they met one or more of the following criteria: examined mild cognitive impairment (MCI) criteria for dementia risk prediction and included measures of sensitivity, specificity or the area under the receiver operating curve (AUC); examined dementia risk prediction in models constructed for application in the whole population without dementia and included measurements of sensitivity, specificity or the AUC; reviewed dementia risk screening in older adults (individuals aged ≥65 years).

Articles were excluded from review if they met one or more of the following criteria: comprised a cross-sectional or case.control study; included a test sample that was restricted at baseline by cognitive criteria other than dementia status (for example, a minimum Mini-Mental State Examination score < 22); for population models only, assessed the predictive utility of various factors, such as candidate biomarkers derived from neuroimaging, cerebrospinal fluid neurochemistry or blood plasma, in MCI populations. In relation to the latter criterion, such articles have been the subject of previous reviews and each factor would require a separate article to be adequately assessed.

In total, 1,255 articles were identified for review from database searches. Of these, 141 articles were selected on the basis of their title and abstract for in-depth assessment. In total, 24 articles met the inclusion criteria. These articles covered 7 models of MCI and 17 models designed for testing in the whole older population. An additional population-based model was identified while the manuscript was under peer review, and this has also been included.
___


___
Box 2. Measures of Predictive Accuracy

Sensitivity
The proportion of at-risk cases correctly identified as being at-risk.

Specificity
The proportion of not-at-risk cases correctly identified as being not at risk.

Area under the receiver operating characteristic curve (AuC)
The AUC is a measure of discriminative accuracy and represents the probability that the estimated risk score for a randomly chosen at-risk individual is higher than the estimated risk score for a randomly chosen not-at-risk individual. Thus, an AUC of 1 reflects perfect sensitivity and specificity.

Positive predictive value (PPv)
The proportion of individuals with a positive test result who are correctly classified.

Negative predictive value (NPv)
The proportion of individuals with a negative test result who are correctly classified.
___



Predictive values as well as sensitivity and specificity estimates depend on the prevalence of the disease. As few studies of dementia risk prediction have reported predictive values, we have focused on the other measures of diagnostic accuracy in this Review.


Mild Cognitive Impairment Criteria


The identification of individuals at high risk of dementia has focused on the concept of mild cognitive impairment (MCI), which is considered to be an intermediate state between normal cognitive aging and dementia. Numerous definitions of MCI have been proposed that vary in both the type of impairment that needs to be present (for example, memory versus non-memory impairments) and level of deficit severity that is required for a diagnosis of MCI to be made.[1] The amnestic MCI criteria (A-MCI) proposed by the Mayo Clinic focus on memory impairment and are the most widely applied criteria for MCI.[2] Other common diagnostic classifications of MCI include aging-associated cognitive decline (AACD),[3] which focuses on cognitive deficits that are thought to be a normal consequence of aging, and the Stockholm consensus revised MCI criteria (MCIr),[4] which concentrate on preclinical cognitive impairment that is associated with memory and non-memory deficits. An overview of the various definitions of MCI is presented in Box 3 .


___
Box 3. Mild Cognitive Impairment Criteria

Examples of mild cognitive impairment (MCI) definitions are given below. For each definition, all criteria need to be met for a classification of MCI.

Mayo Clinic amnestic MCi (A-MCi)

* Subjective memory complaint
* Preserved general cognitive function
* Objective evidence of memory impairment*
* Intact functional ability
* Absence of dementia

Stockholm consensus revised MCi (MCir)

* Cognitive complaint (reported by the individual or a family member)
* Absence of dementia
* Change from normal functioning
* Decline in any area of cognitive function
* Preserved overall functioning with the possibility of
* difficulty in activities of daily living

Aging-associated cognitive decline (AACD)

* Patient or informant report of cognitive decline (must be described as gradual and present for at least 6 months)
* Patient and/or informant report of memory impairment
* Impairment in cognition in any one of the following five domains: memory and learning, attention and concentration, thinking, language, and visuospatial functioning
* Impairment is not due to a past or present medical condition or psychoactive substance use that might interfere with cognitive function

*Mayo Clinic definitions vary depending on the presence of memory impairment. For a nonamnestic MCI diagnosis, nonmemory cognitive domains exhibit deficits but memory itself is preserved. Where both memory and non-memory domains are impaired, the term multiple domain MCI is applied. All other MCI criteria must be fulfilled.
___



The annual conversion rate from MCI to dementia varies from 5% to 10% according to the definition of MCI used at diagnosis.[5,6] Moreover, some studies have reported that 40-70% of MCI cases remain stable or revert to normal cognitive function over time.[5,7-9] Thus, although the incidence of dementia is generally higher in individuals with MCI (5-10%) than in the general population (4.4%),[10] the fact that a large number of people with MCI do not have a progressive condition has raised questions about the reliability of MCI as a population screening tool.

In population-based samples, A-MCI and other Mayo Clinic-defined MCI criteria generally exhibit low sensitivity but moderate to high specificity for the identification of individuals at risk of dementia.[11-16] For A-MCI, studies have shown that despite variation in sample size, age groups, severity thresholds, follow-up periods (2-9 years) and inclusion criteria modifications (such as removal of the requirement to have a subjective memory complaint [SMC] to make a diagnosis), sensitivity estimates remain ≤20%, while specificity values are always >90% (Supplementary Table 1 online).[11-14,16] These results suggest that A-MCI is too restrictive to be used for dementia risk screening in the population. Indeed, only a small proportion of at-risk individuals will meet these diagnostic criteria, leaving a high prevalence of such cases that will be incorrectly classified as being not at risk. By contrast, individuals who are not at risk of developing dementia will be reliably identified. The discriminative accuracy of A-MCI across studies has ranged from poor to fair (AUC 0.48-0.59).[11-14,16]

The expanded Mayo Clinic definitions of MCI—multiple domain MCI (where individuals are required to show impairment in memory and non-memory domains), revised MCI (where individuals are required to show impairment in one or more cognitive domains) and combined MCI (where individuals satisfying criteria across various MCI definitions are collapsed into a single category)—show higher levels of sensitivity (generally >30%) than does A-MCI, while also exhibiting moderate to high specificity (generally >70%).[12,15] As with A-MCI, the discriminative accuracy shown by these models ranges from poor to fair (AUC 0.48-0.74).[12,15] Taken together, studies of the Mayo Clinic MCI definitions indicate that assessment of multiple cognitive domains provides a higher level of diagnostic accuracy than do criteria that are based on one domain alone.

Variable estimates of sensitivity and specificity have been reported for non-Mayo Clinic-defined classifications of MCI, including AACD,[3] MCIr (defined as impaired general cognitive function, declined performance in one or more cognitive domains, and essentially normal functional ability),[4] cognitive impairment (defined as a deficit in one or more cognitive domains with or without patient or informant reporting of memory loss),[14] and MCI based on a Mini Mental State Examination (MMSE) scor ≤26.[17] In general, these non-Mayo Clinic models exhibit sensitivity estimates >60%, specificity values that range from low to high, and diagnostic accuracy that ranges from poor to good (AUC 0.26-0.80; Supplementary Table 1 online).[11,13,14,16,18] Of note, an MMSE score ≤26 was found to be as good as, or better than, most other classifications of MCI for the prediction of dementia, exhibiting 78% sensitivity, 70% specificity and an AUC of 0.74.[18] This finding questions whether the diagnostic accuracy observed with some expanded MCI definitions is sufficiently higher than the accuracy observed with the MMSE to justify the additional costs incurred by the in- depth data collection that is required with the former.

Overall, no MCI criteria at any cognitive severity level, follow-up interval, or age limits have both sensitivity and specificity estimates >80%. Furthermore, despite being tested in only one study, the assumption that certain subtypes of MCI are associated with an increased risk of specific dementia subtypes (for example, A-MCI with Alzheimer disease [AD]) seems to be unfounded.[12] Most definitions of MCI have moderate to high specificity but low sensitivity and, thus, fail to identify many individuals at risk of future dementia. Research is now focused on the development of methods to differentiate MCI cases that represent a prodromal stage of dementia from cases that reflect a benign change in cognitive decline. Thus, the distinction has been drawn between progressive and non-progressive MCI on the basis of the probability of progression to dementia.

MCI criteria have been supplemented with clinical observations (for example, neuropsychiatric screening),[19-22] neuroimaging (for example, structural markers of brain integrity such as hippocampal, ventricular and medial temporal lobe atrophy)[7,13,23-26] and biological measures (for example, elevation of the 42 amino acid form of amyloid-β in cerebrospinal fluid or reduction of platelet amyloid precursor protein isoform levels).[27-33] The sensitivities and specificities of neuropsychological and non-neuropsychological measures in the prediction of dementia vary from low to high, but rarely do both estimates for any one measure reach the top of this range. Furthermore, the ability of such measures to increase the accuracy of MCI criteria for the prediction of dementia and dementia subtypes (for example, AD or vascular dementia) in population-representative samples remains to be tested.


Population Screening Models


Models of dementia risk prediction have been developed for use across the whole older population. Such models avoid the controversy surrounding MCI, as no assumption is made about the underlying risk of future dementia. The aim of population screening models is to classify individuals into various risk categories (such as low, moderate and high) so that decision rules can be followed. For example, a low-risk diagnosis would enable reassurance in the decision not to intervene and provide an opportunity to minimize anxiety in an individual or their family, particularly in people who perceive their risk of dementia to be high. A diagnosis of moderate risk would prompt re-examination (such as further neuropsychological, neuroimaging or laboratory work-up) or a period of watchful waiting, while a diagnosis of high risk would trigger intervention, such as risk factor manipulation.[34]

Dementia risk models have mainly drawn on data from neuropsychological testing and sociodemographic variables, including age, sex and education. These models can be broadly divided into the following categories: cognitive profiles (descriptions of the various cognitive tests used in such models are outlined in Table 1 );[35-44] health and vascular risk indices;[45,46] multifactorial models typically combining demographic and neuropsychological measures with health or genetic variables;[34,47,48] multistage screening approaches;[49] and multi-dementia subtype preclinical groupings.[50] Details of the methods and design of each dementia risk model study are presented in Supplementary Table 2 online.


___
Table 1. Neuropsychological Tests

Activity recall (of all tasks completed during the testing session)
- Learning and memory (recall)

Animal naming task
- Verbal fluency

Buschke Selective Reminding Test (SRT)
- Verbal learning and memory

Cambridge Cognitive Examination (CAMCOG)
- Comprises 14 composite subtest scores examining orientation, comprehension, naming, fluency, definitions, memory, picture recognition, general knowledge, attention, copying, ideomotor praxis, abstraction, visual perception and miscellaneous

Cognitive Abilities Screenin Instrument (CASI)
- Comprises nine subscales covering attention, concentration or mental manipulation, orientation, episodic memory, semantic memory, language abilities, visual construction, list-generating fluency, abstraction and judgment

Consortium to Establish a Registry for Alzheimer's Disease Word List Delayed Recall test (CERAD-WLDR)
- Memory

Digit Letter Task (DL)
- Attention and executive function

Digit Symbol Substitution Test from the Wechsler Adult Intelligence Scale-revised
- Attention and executive function

Fuld Object Memory Evaluation (FOME)
- Memory

Free and Cued Selective Reminding Test (FCSRT)
- Memory (verbao)

Identical Pictures Test
- Attention

Intra-categorical Delayed Selective Reminding Test 7 (IDSR-7)
- Episodic memory

Memory for text
- Learning and memory (recall)

Mini-Mental State Examination (MMSE)
- General cognitive function

Modified Mini-Mental State Examination (3MSE)
- General cognitive function

Rey Auditory-Verbal Learning Test (RAVLT
- Memory (delayed recall)

Structured Interview for the Diagnosis of Dementia of Alzheimer type, Multi-infarct Dementia and Dementia of other Etiology according to the International Statistical Classification of Diseases and Related Health Problems Tenth Revision and the Diagnostic and Statistical Manual of Mental Disorders Fourth Edition (SIDAM)
- Cognitive test covering the following areas of neuropsychological functioning: orientation, memory, aphasia and apraxia, as well as intellectual, verbal and constructional abilities

Trail Making Test Part A (TMT-A)
- Executive function

Trail Making Test Part B (TMT-B)
- Visual tracking, mental flexibility and attention

Vocabulary (total score) from the Wechsler Adult Intelligence Scale-Revised (WAIS-R vocabulary)
- Verbal IQ

Wechsler Memory Scale: Information subtest (WMS)
- Memory

Wechsler Paired-Associate Learning Task (PAL)
- Cued recall
___



Cognitive Profiles

Single and Multiple Test Models. Tests of delayed recall, global cognitive function, perceptual speed and executive performance have been suggested to be relatively good predictors of future dementia, but predictive ability of such tests has rarely been validated.[51] In a study of the Cognitive Abilities Screening Instrument (CASI) in individuals aged ≥70 years, the total CASI score and the CASI subscale scores for episodic memory, orientation, list-generating fluency and language were found to the best single neuropsychological predictors of dementia onset within 3 years (AUC 0.59-0.74).[38] For longer periods to dementia onset (3-6 years), a smaller number of single factors (the total and episodic memory subscale CASI scores, as well as SMCs) predicted future dementia with comparable accuracy (AUC 0.62-0.66). Combinations of test scores or the inclusion of demographic characteristics led to only marginal improvements in diagnostic accuracy. In this study, the CASI score was used to refer individuals for clinical assessment, with people exhibiting scores <74 at baseline being excluded, as such scores are considered to be within the range of dementia. Thus, the use of CASI scores for dementia prediction might have led to a circularity effect and, hence, the diagnostic properties of CASI might have been overestimated.

A three-test model comprising an animal naming task, the Intra-categorical Delayed Selective Reminding Test 7 (IDSR-7) and the Trail Making Test Part B (TMT-B) showed moderate predictive ability (ranging from 17% to 99% sensitivity and from 26% to 97% specificity with various cut-off scores) and good diagnostic accuracy (AUC 0.83) for possible or probable AD in individuals aged 75 years at baseline who were followed up at 2.5 years.[39] Classification accuracy was unchanged when the TMT-B was excluded from this model, suggesting that the combination of the two other psychometric instruments could be effective for AD risk screening. In the simplified model, however, the cut-off score with the highest sensitivity (91%) showed low specificity (56%).

A model comprising an animal naming task, two memory tests (the Rey Auditory-Verbal Learning Test [RAVLT] and the Wechsler Memory Scale III[52]), age and education level showed similar diagnostic accuracy to the three-test model described above[39] but over a longer period (5 years) to disease onset.[44] In this study, conducted in individuals aged ≥65 years, the model predicted possible or probable AD with 74% sensitivity, 83% specificity and an AUC of 0.83. Inclusion of apolipoprotein E (APOE) ε4 status did not improve the predictive accuracy of this model. At 10 years' follow-up of the same cohort, a model that included age, education and only one cognitive test (RAVLT) predicted possible or probable AD with 73% sensitivity and 70% specificity (AUC 0.77).[44] In contrast to the 5 year model, inclusion of APOE ε4 status resulted in a marked improvement in predictive accuracy. Of note, although predictive accuracy seems to be higher for multiple-cognitive-test models than for single-test models, direct comparisons between these two approaches are not possible, as they each used different outcome measures (namely, possible or probable AD in multiple-test models versus all-cause dementia in single-test models).

The most complex cognitive profile models have comprised four cognitive tests. The first of these models to be developed used the Buschke Selective Reminding Test (delayed recall score), a verbal fluency task (four trials, each lasting 60 s, requiring naming of fruits, animals, flowers and vegetables), the Digit Symbol Substitution Test (DSST), and the recall score from the Fuld Object Memory Evaluation.[40] This model exhibited high specificity (94%) but low sensitivity (50%) for the prediction of dementia in individuals aged 75-85 years who were followed up for at least 4 years (the AUC was not reported in this study). Furthermore, this model was better at predicting dementia risk in individuals with a short period to disease onset (median 2.6 years) than in people with a longer delay to the appearance of symptoms (median 4.6 years).

The second four-cognitive-test model to be developed combined the Word Delayed Recall Test (using three words), a word fluency test, the TMT-B, and the CASI abstraction-judgment subscale score, and was used to assess individuals who had a Clinical Dementia Rating (CDR) score of 0 or 1.[41] This model exhibited high sensitivity (93%) and low specificity (66%) for the prediction of 5 year dementia in individuals aged ≥65 years (AUC 0.88),[41] but did not outperform the MMSE in this sample (sensitivity 81%, specificity 65% and AUC 0.81). Moreover, the predictive ability of the four-test model was not improved with adjustment of the limits on age (that is, to restrict the sample to individuals aged 70-79 years) or the test outcome (to AD or mixed AD and vascular dementia, rather than dementia), or reduction of the model to include only tests of memory.

Finally, the optimal model derived from the Cambridge Cognitive Examination (CAMCOG) for predicting AD 2 years before disease onset combined age with 4 of the 14 CAMCOG subscale scores (verbal fluency, memory, general knowledge and attention-calculation).[42] This model exhibited 69% sensitivity and 84% specificity. The specificity shown by this model declined marginally to 80% (sensitivity 69%) when dementia was used as the outcome. The predictive accuracy (AUC value) of the CAMCOG and its subscale scores was not reported.

Factor Scores. An attention and executive function factor score, which combined the results of the TMT-B, the Digit Letter Test, the DSST and the Identical Pictures Test, exhibited a higher predictive accuracy for AD than did the MMSE in a study of individuals aged 70-103 years who were followed up at 4 years (AUC 0.86 versus 0.68).[43] Furthermore, the TMT-B (AUC 0.88) and the Identical Pictures Test (AUC 0.88), were both more accurate than the MMSE for AD prediction. These results suggest that executive dysfunction occurs early in the pathogenesis of AD. In contrast to the attention and executive function factor score, a learning and recall factor score, which was derived from the results of the Paired Associated Learning task, a memory for text task and an activity recall test, did not outperform the MMSE (the learning and recall factor score had an AUC of 0.72).[43] The report's authors suggested that the low diagnostic accuracy of the learning and recall factor score resulted from the lack of a delayed recall measure. No sensitivity and specificity estimates were reported for the attention and executive function or the learning and recall factor scores.

Within-person Across-test Variability. Knowledge of how much a person's performance varies across different neuropsychological tests might improve the accuracy of dementia risk prediction.[37] A model incorporating demographic factors (age, sex and education), health variables and absolute performance on several neuropsychology measures (the vocabulary test and the DSST from the Wechsler Adult Intelligence Scale-Revised, and the free recall score from the Free and Cued Selective Reminding Test), showed 85% sensitivity (for a set specificity of 80%) for dementia prediction in individuals aged ≥70 years who were followed up for 1 year. This sensitivity estimate increased to 88% when a marker of within-person across-test variability was added to the model.[37] Marked variation in test performance is thought to reflect impairment of top-down executive processes, which are controlled by the frontal cortical regions that are damaged in the early stages of disease. The predictive accuracy of the model described above is relatively high and the inclusion of other tests or risk factors might further improve its predictive capability. Whether classification accuracy will remain as high with changes in the age of the sample (for example, with extension of the lower age boundary to 65 years) and longer time frames needs to be explored.

Change Scores. Reliable change indices (RCI) are used to estimate the probability that changes in test scores across test sessions are attributable to a change in an individual's ability to perform a task, rather than measurement error, practice effects or regression-to-the-mean effects. Whether RCI can be used to predict dementia has been tested for scores generated from the MMSE[36] and SIDAM (Structured Interview for Diagnosis of Dementia of Alzheimer type, Multi-infarct Dementia and Dementia of other Etiology according to the International Statistical Classification of Diseases and Related Health Problems Tenth Revision and the Diagnostic and Statistical Manual of Mental Disorders Fourth Edition).[35] In these studies, the cognitive criteria used to make the diagnosis of dementia included information from the MMSE and SIDAM. Thus, given that the tests used to predict dementia were not entirely independent of the outcome measure (that is, dementia status), the diagnostic accuracies of the MMSE and SIDAM might have been overestimated. The results of these studies showed that although individuals who developed dementia demonstrated a marked deterioration in the MMSE and SIDAM scores, the RCI scores exhibited low predictive accuracy for dementia over 1.5 year intervals in individuals aged ≥75 years (mean total follow-up 5.6 years).[35,36]

The poor results for RCI might be explained by several factors. First, RCI derived from a neuropsychological test that is a poor predictor of dementia would also be expected to show poor predictive accuracy. Second, as highlighted by the authors of the MMSE and SIDAM reports,[35,36] the predictive accuracy of RCI might depend on the rate of patient decline. Indeed, individuals who have a slow rate of decline might not be captured by neuropsychological tests if the testing intervals are too close together or if variation due to disease cannot be distinguished from normal variability. Whether RCI could be used for dementia risk screening in populations is questionable. RCI require complete data sets from at least two assessments and, hence, incur added testing costs that might offset any advantages gained from such indices. Furthermore, despite the added information that is gained from the requirement of multiple test sessions to calculate RCI, the predictive accuracy of these indices does not exceed that of single or multiple psychometric test models.

Health-based Models

A model derived from demographic (age, sex and education), lifestyle (physical activity) and midlife health risk factors (systolic blood pressure, BMI and total cholesterol level) had moderate diagnostic accuracy for the prediction of dementia over a follow-up period of 20 years (AUC 0.77; sensitivity 77% and specificity 63% with a cut-off score ≥9; sensitivity 63% and specificity 75% with a cut-off score ≥10). The addition of APOE ε4 status resulted in a marginal improvement in the predictive capability of the model (AUC 0.78),[45] while the exclusion of age resulted in a slight decrease in discriminative accuracy (AUC 0.76).

A model created by combining sex with a comprehensive vascular risk index had moderate predictive accuracy for 10 year incident dementia (AUC 0.75), but showed poor accuracy at a 20 year follow-up (AUC 0.66).[46] Cut-off scores with estimates of sensitivity and specificity were not reported in this study.

Despite the lengthy follow-ups, the predictive accuracy of health-based models is similar to that of neuro-psychological models derived from data obtained in later life. Importantly, models based on health would allow risk factor manipulation to be undertaken before the influence of age and disease comorbidity (through polypharmacy and other effects) limit intervention options. Detailed health assessments, however, are labor-intensive and expensive. Thus, before any recommendations could be made for in-depth health screening of the entire middle-aged population to determine dementia risk, the cost-effectiveness of this approach would need to be proved. This assertion does not diminish the importance of existing health screening programs to reduce the morbidity and mortality associated with diseases such as hypertension and diabetes.

Multifactor Models

Multifactor models have mainly drawn on data from neuropsychological testing, health screening, neuro-imaging, genetics and informant or patient reports of memory or cognitive difficulties. The most complex of these models is the Late-Life Dementia Risk Index,[34] which assesses the following factors: age, cognition (measured by the Modified Mental State Examination [3MSE] and DSST), BMI, APOE ε4 status, evidence from MRI of white matter disease or ventricular enlargement, vascular health (presence of internal carotid artery thickening on ultrasound or a history of bypass surgery), physical function, and lifestyle (alcohol use). This index had a c statistic of 0.81 for 6 year dementia in a population sample aged ≥65 years. Risk of incident dementia was <5% in the low-risk group (index score range 0-3), <25% in the moderate-risk group (index score range 4-7) and ≈50% in the high-risk group (index score ≥8).

A revised version of the Late-Life Dementia Risk Index—the Brief Dementia Risk Index—has recently been reported.[53] The revised index combines demographic (age), cognitive (delayed recall test, figure copying, verbal instruction performance, animal naming, and self-reported cognitive difficulties), lifestyle (alcohol use) and medical factors (stroke, peripheral artery disease, a history of bypass surgery, and BMI). Discriminative accuracy was markedly higher for the full index than for the revised version (c statistic 0.77); however, risk stratification was found to be essentially unchanged (risk of incident dementia 4%, 24% and 52% in the low-risk, moderate-risk and high-risk groups, respectively).[53]

A second multifactor model, which combined measures of episodic memory and motor speed (that is, by use of the Consortium to Establish a Registry for Alzheimer's Disease Word List Delayed Recall test battery and TMT-A) with APOE ε4 status and level of subjective memory decline, showed high predictive accuracy for probable AD at 5 years in individuals aged 75 years (AUC 0.91).[48] The most parsimonious multifactor model, which was designed to include only variables that are easily obtained outside the clinic (the model comprised age, the 3MSE score, informant report of memory loss and family history of dementia), had low sensitivity (27%) but high specificity (94%) for dementia prediction at 5 years.[47] The sensitivity estimate for this model improved when the outcome measure was restricted to prediction of AD (sensitivity 45% and specificity 89%). When family history of dementia was removed, the model exhibited moderate sensitivity (79%) and poor specificity (56%) for dementia prediction. Substitution of the 3MSE with the MMSE resulted in a marginal loss of sensitivity to 73% but an increase in specificity to 68%.[47]

Multistage Screening

A three-step screening procedure that consists of a single question about memory problems, followed by assessment of global cognitive function by means of the MMSE and neuropsychological testing (episodic recall, verbal fluency and visuospatial tasks) has been proposed to replicate clinic-based high-risk targeted case finding for AD or dementia. Nevertheless, despite an increase in predictive accuracy with increasing sample restriction, the sensitivity estimate after each step only showed a marginal change (sensitivity estimates 21-51% for the whole population, 25-40% for individuals with SMCs, and 48-56% for individuals with SMCs and proven cognitive impairment). The specificity estimates with each step were largely unchanged and generally exceeded 70% (70-94% for the whole population, 84-89% for individuals with SMCs, and 46-92% for individuals with SMCs and proven cognitive impairment).[49]

Multigroup Classifications

Given that pure dementia subtypes are rarely observed, an at-risk classification that combines deficits associated with AD, vascular dementia, and Parkinson disease and related dementias (dementia associated with Lewy bodies) might improve the predictive accuracy of screening models. Nevertheless, a broad model that defined a risk state by collapsing across seven different preclinical subgroups, and was operationalized using a combination of cognitive, motor and vascular features, showed low specificity (35.7%) and high sensitivity (89.7%) for dementia risk prediction after 3 years.[50] The results of this study indicate that this classification scheme is too all-encompassing to be used for population dementia risk screening, as too many people labeled as having a preclinical syndrome did not progress to dementia. The study's investigators argued that additional information (for example, from further neuropsychological testing, and neuroimaging markers of AD and vascular pathology) could refine the preclinical subgroupings and, hence, improve the model. However, given that a considerable amount of resources were already needed to diagnose the preclinical conditions, the screening model described above is unlikely to be cost-effective. Whether each preclinical condition has high diagnostic accuracy for the prediction of specific dementia subtypes remains to be tested.


Conclusions from Population Screening Models

Few dementia risk models have a high level of predictive accuracy. The best models have incorporated diverse sources of information across multiple factors, including neuropsychological testing and health screening. Poor accuracy is typically associated with single-factor models, longer follow-up intervals and outcomes that are restricted to all-cause dementia rather than AD. In general, most models exhibit either high sensitivity or high specificity, rather than having high values for both measures. The required accuracy of a model will depend on the reason for diagnostic screening. For example, if high-risk cases are to be exposed to a potentially dangerous intervention, predictive accuracy would need to be high, whereas if screening is to be used to determine additional follow-up, predictive accuracy could be lower.

As a result of differences in study design, risk factor operationalization, follow-up times, dementia outcomes (possible or probable AD versus all-cause dementia) and study populations (for example, age differences), the pooling of data from various studies is not possible, and external validation would be difficult. Indeed, no model has been validated externally, and few have undergone rigorous statistical testing (for example, validation, calibration or sensitivity analysis across factors such as age and education). Moreover, no uniform approach exists to provide common standards for risk factor selection and analyses in the development of an algorithm to predict dementia risk. Empirical comparisons of the various methodologies need to be conducted to assess their relative merits, and to determine the most accurate approaches across diverse populations (that is, populations that differ in culture, language and educational attainment) and settings (that is, in the clinic versus in the general population). Furthermore, all models need to be examined for their practical application, including assessment of implementation feasibility and analysis of cost-effectiveness.


Analytical Considerations


The predictive accuracy of dementia risk models has been evaluated with measures of sensitivity and specificity from AUC analysis. These measures have their limitations. Sensitivity and specificity estimates are influenced by several factors, including disease prevalence, intrinsic biological variability of disease across individuals, the severity or temporal stage of disease at the time of testing, and the specific nature of the test population used to initially construct the model (for example, clinic versus general population samples).[54,55] No studies of dementia risk prediction that have been reviewed here have acknowledged that the reported sensitivity and specificity estimates might not extrapolate to all samples and settings. Indeed, the diagnostic accuracies of MCI and population screening models are probably overestimated in the studies explored in this Review, particularly in studies where the sample size was small and disease incidence was high, as these scenarios increase the probability of detecting cases of disease. Estimates of sensitivity and specificity will also vary depending on how the 'best' cut-off score was selected (that is, to maximize the combination of sensitivity and specificity, or to achieve the highest estimate of sensitivity alone).

Before any model can be recommended for dementia risk prediction, external validation must be conducted to ensure that a model can be applied outside the sample from which it was derived. Furthermore, no model to date has provided details on the speed of dementia progression in at-risk individuals and the probability of backward transition (that is, from an at-risk state to a not-at-risk category). This information will be particularly important where risk models are used to guide clinical decisions, and to inform individuals of the duration of their given risk state (for example, whether progression of disease will be fast or slow in at-risk cases) or the probability of moving to a different risk category (for example, from cognitive decline to disease progression or improvement).


Conclusions and Future Directions


Early identification of individuals who have a high risk of dementia will be of great importance when strategies to prevent or delay dementia onset become available. Before recommendations on the use of prognostic tools in the general population can be made, the ethical, economic and other consequences of such tools must be considered. Whether the application of such tools can enable natural disease progression to be altered remains to be explored, and clear guidelines on the outcome of risk identification will need to be developed. Methods are required that reliably measure an individual's level of dementia risk over a specified time frame. The construction of effective predictive models necessitates a greater understanding than currently exists of the natural course of dementia and the risk factors associated with this condition. This goal will require an in-depth analysis of the large amount of data already available from population-based longitudinal dementia studies and the results from investigations using newly advanced techniques (for example, diffusion tensor imaging). The primary focus of future research, however, needs to be on economically viable methods rather than on, for example, expensive neuroimaging or serum analysis, which should both only be applied to methods if cost-effective. Various dementia risk screening approaches will probably be necessary depending on the outcome to be predicted (that is, the subtype of dementia), the nature of the sample (age group, educational attainment, and country of origin), and the length of the follow-up interval.


Key Points

* Strategies are needed to identify individuals at risk of dementia long before disease onset, so that resources, treatment and prevention efforts can be efficiently targeted
* Numerous models of dementia risk have been proposed but their relative merits in terms of predictive accuracy are unknown
* Our evaluation of the literature suggests that current risk models are poor at distinguishing people at risk of developing dementia from not-at-risk individuals
* A consensus model for the prediction of dementia is needed that is applicable to both population and clinical settings


References

1. Matthews, F. E., Stephan, B. C., Bond, J., McKeith, I. & Brayne, C. Operationalisation of mild cognitive impairment: a graphical approach. PLoS Med. 4, 1615-1619 (2007).
2. Petersen, R. C. et al. Current concepts in mild cognitive impairment. Arch. Neurol. 58, 1985-1992 (2001).
3. Levy, R. Aging-associated cognitive decline. Working Party of the International Psychogeriatric Association in collaboration with the World Health Organization. Int. Psychogeriatr. 6, 63-68 (1994).
4. Winblad, B. et al. Mild cognitive impairment— beyond controversies, towards a consensus: report of the International Working Group on Mild Cognitive Impairment. J. Intern. Med. 256, 240-246 (2004).
5. Mitchell, A. J. & Shiri-Feshki, M. Rate of progression of mild cognitive impairment to dementia—meta-analysis of 41 robust inception cohort studies. Acta Psychiatr. Scand. 119, 252-265 (2009).
6. Matthews, F. E., Stephan, B. C., McKeith, I. G., Bond, J. & Brayne, C. Two-year progression from mild cognitive impairment to dementia: to what extent do different definitions agree? J. Am. Geriatr. Soc. 56, 1424-1433 (2008) .
7. Farias, S. T., Mungas, D., Reed, B. R., Harvey, D. & DeCarli, C. Progression of mild cognitive impairment to dementia in clinic- vs community-based cohorts. Arch. Neurol. 66, 1151-1157 (2009) .
8. Ganguli, M., Dodge, H. H., Shen, C. & DeKosky, S. T. Mild cognitive impairment, amnestic type: an epidemiologic study. Neurology 63, 115-121 (2004).
9. Stephan, B. C., Brayne, C., McKeith, I. G., Bond, J. & Matthews, F. E. Mild cognitive impairment in the older population: who is missed and does it matter? Int. J. Geriatr. Psychiatry 23, 863-871 (2008).
10. Matthews, F. et al. The incidence of dementia in England and Wales: findings from the five identical sites of the MRC CFA Study. PLoS Med.2, e193 (2005).
11. Artero, S., Petersen, R., Touchon, J. & Ritchie, K. Revised criteria for mild cognitive impairment: validation within a longitudinal population study. Dement. Geriatr. Cogn. Disord. 22, 465^470 (2006).
12. Baars, M. A. et al. Predictive value of mild cognitive impairment for dementia. The influence of case definition and age. Dement. Geriatr. Cogn. Disord. 27, 173-181 (2009).
13. Busse, A., Bischkopf, J., Riedel-Heller, S. G. & Angermeyer, M. C. Mild cognitive impairment: prevalence and incidence according to different diagnostic criteria: results of the Leipzig Longitudinal Study of the Aged (LEILA75+). Br.J. Psychiatry 182, 449-454 (2003).
14. Busse, A., Bischkopf, J., Riedel-Heller, S. G. & Angermeyer, M. C. Subclassifications for mild cognitive impairment: prevalence and predictive validity. Psychol. Med. 33, 1029-1038 (2003).
15. Busse, A., Hensel, A., Guhne, U., Angermeyer, M. C. & Riedel-Heller, S. G. Mild cognitive impairment: long-term course of four clinical subtypes. Neurology 67, 2176-2185 (2006).
16. Ritchie, K., Artero, S. & Touchon, J. Classification criteria for mild cognitive impairment: a population-based validation study. Neurology56, 37-42 (2001).
17. Folstein, M. F., Folstein, S. E. & McHugh, P R. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J. Psychiatr. Res. 12, 189-198 (1975).
18. Busse, A., Bischkopf, J., Riedel-Heller, S. G. & Angermeyer, M. C. Mild cognitive impairment: prevalence and predictive validity according to current approaches. Acta Neurol. Scand. 108, 71-81(2003).
19. Bidzan, L., Pachalska, M. & Bidzan, M. Predictors of clinical outcome in MCI. Med. Sci. Monit. 13, CR398-CR405 (2007).
20. Palmer, K. et al. Predictors of progression from mild cognitive impairment to Alzheimer disease. Neurology 68, 1596-1602 (2007).
21. Ravaglia, G. et al. Conversion of mild cognitive impairment to dementia: predictive role of mild cognitive impairment subtypes and vascular risk factors. Dement. Geriatr. Cogn. Disord. 21, 51-58 (2006).
22. Tabert, M. H. et al. Functional deficits in patients with mild cognitive impairment: prediction of AD. Neurology 58, 758-764 (2002).
23. Fleisher, A. S. et al. Volumetric MRI vs clinical predictors of Alzheimer disease in mild cognitive impairment. Neurology 70, 191-199 (2008).
24. Hirao, K. et al. The prediction of rapid conversion to Alzheimer's disease in mild cognitive impairment using regional cerebral blood flow SPECT. Neuroimage 28, 1014-1021 (2005).
25. Kantarci, K. et al. Risk of dementia in MCI: combined effect of cerebrovascular disease, volumetric MRI, and 1H MRS. Neurology 72, 1519-1525 (2009).
26. McKelvey, R. et al. Lack of prognostic significance of SPECT abnormalities in non-demented elderly subjects with memory loss. Can. J. Neurol. Sci. 26, 23-28 (1999).
27. Herukka, S. K., Pennanen, C., Soininen, H. & Pirttila, T. CSF A342, tau and phosphorylated tau correlate with medial temporal lobe atrophy. J.Alzheimers Dis. 14, 51-57 (2008).
28. Hsiung, G. Y., Sadovnick, A. D. & Feldman, H. Apolipoprotein E e4 genotype as a risk factor for cognitive decline and dementia: data from the Canadian Study of Health and Aging. CMAJ 171, 863-867 (2004).
29. Hyman, B. T. et al. Apolipoprotein E and cognitive change in an elderly population. Ann. Neurol. 40, 55-66 (1996).
30. Mattsson, N. et al. CSF Biomarkers and incipient Alzheimer disease in patients with mild cognitive impairment. JAMA 302, 385-393 (2009).
31. Mitchell, A. J. CSF phosphorylated tau in the diagnosis and prognosis of mild cognitive impairment and Alzheimer's disease: a meta-analysis of 51 studies. J. Neurol. Neurosurg. Psychiatry 80, 966-975 (2009).
32. Riemenschneider, M. et al. Cerebrospinal fluid tau and -amyloid 42 proteins identify Alzheimer disease in subjects with mild cognitive impairment. Arch. Neurol. 59, 1729-1734 (2002).
33. Tang, B. L. & Kumar, R. Biomarkers of mild cognitive impairment and Alzheimer's disease. Ann. Acad. Med. Singapore 37, 406^416 (2008).
34. Barnes, D. E. et al. Predicting risk of dementia in older adults: the late-life dementia risk index. Neurology 73, 173-179 (2009).
35. Hensel, A., Angermeyer, M. C. & Riedel-Heller, S. G. Measuring cognitive change in older adults. Do reliable change indices of the SIDAM predict dementia? J. Neurol. 254, 1359-1365 (2007).
36. Hensel, A. et al. Does a reliable decline in Mini Mental State Examination total score predict dementia? Diagnostic accuracy of two reliable change indices. Dement. Geriatr. Cogn. Disord.27, 50-58 (2009).
37. Holtzer, R., Verghese, J., Wang, C., Hall, C. B. & Lipton, R. B. Within-person across-neuropsychological test variability and incident dementia. JAMA 300, 823-830 (2008).
38. Jorm, A. F., Masaki, K. H., Petrovitch, H., Ross, G. W. & White, L. R. Cognitive deficits 3 to 6 years before dementia onset in a population sample: the Honolulu-Asia aging study. J. Am. Geriatr. Soc. 53, 452-455 (2005).
39. Jungwirth, S. et al. Screening for Alzheimer's dementia at age 78 with short psychometric instruments. Int. Psychogeriatr. 21, 548-559 (2009).
40. Masur, D. M., Sliwinski, M., Lipton, R. B., Blau, A. D. & Crystal, H. A. Neuropsychological prediction of dementia and the absence of dementia in healthy elderly persons. Neurology44, 1427-1432 (1994).
41. Nakata, E. et al. Combined memory and executive function tests can screen mild cognitive impairment and converters to dementia in a community: the Osaki-Tajiri project. Neuroepidemiology 33, 103-110 (2009).
42. Nielsen, H., Lolk, A., Andersen, K., Andersen, J. & Kragh-S0rensen, P Characteristics of elderly who develop Alzheimer's disease during the next two years-a neuropsychological study using CAMCOG. The Odense Study. Int. J. Geriatr. Psychiatry 14, 957-963 (1999).
43. Rapp, M. A. & Reischies, F. M. Attention and executive control predict Alzheimer disease in late life: results from the Berlin Aging Study (BASE). Am. J. Geriatr. Psychiatry 13, 134-141 (2005).
44. Tierney, M. C., Yao, C., Kiss, A. & McDowell, I. Neuropsychological tests accurately predict incident Alzheimer disease after 5 and 10 years. Neurology 64, 1853-1859 (2005).
45. Kivipelto, M. et al. Risk score for the prediction of dementia risk in 20 years among middle aged people: a longitudinal, population-based study. Lancet Neurol. 5, 735-741 (2006).
46. Mitnitski, A. et al. A vascular risk factor index in relation to mortality and incident dementia. Eur. J. Neurol. 13, 514-521 (2006).
47. Hogan, D. B. & Ebly, E. M. Predicting who will develop dementia in a cohort of Canadian seniors. Can. J. Neurol. Sci. 27, 18-24 (2000).
48. Jungwirth, S. et al. Prediction of Alzheimer dementia with short neuropsychological instruments. J. Neural Transm. 116, 1513-1521 (2009).
49. Palmer, K., Backman, L., Winblad, B. & Fratiglioni, L. Detection of Alzheimer's disease and dementia in the preclinical phase: population based cohort study. BMJ 326, 245 (2003).
50. Waite, L. M., Broe, G. A., Grayson, D. A. & Creasey, H. Preclinical syndromes predict dementia: the Sydney older persons study. J. Neurol. Neurosurg. Psychiatry 71, 296-302 (2001).
51. Backman, L., Jones, S., Berger, A. K., Laukka, E. J. & Small, B. J. Multiple cognitive deficits during the transition to Alzheimer's disease. J. Intern. Med. 256, 195-204 (2004).
52. Wechsler, D. Manual for the Wechsler Adult Intelligence Scale—Revised (The Psychological Corporation, New York, 1981).
53. Barnes, D. E. et al. Commentary on "Developing a national strategy to prevent dementia: Leon Thal Symposium 2009." Dementia risk indices: a framework for identifying individuals with a high dementia risk. Alzheimers Dement. 6, 138-141 (2010).
54. Cook, N. R. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 115, 928-935 (2007).
55. Cook, N. R. Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve. Clin. Chem. 54, 17-23 (2008).

___
Disclaimer

The material presented here does not necessarily reflect the views of Medscape, LLC or companies that support educational programming on www.medscapecme.com. These materials may discuss therapeutic products that have not been approved by the US Food and Drug Administration and off-label uses of approved products. A qualified healthcare professional should be consulted before using any therapeutic product discussed. Readers should verify all information and data before treating patients or employing any therapies described in this educational activity.
Acknowledgments

Désirée Lie, Univesity of California, Orange, CA, is the author of and is solely responsible for the content of the learning objectives, questions and answers of the MedscapeCME-accredited continuing medical education activity associated with this article.
Reprint Address

B. C. M. Stephan bcms2@medschl.cam.ac.uk

Nat Rev Neurol. 2010;6(6) © 2010 Nature Publishing Group