In comparing the constructs of two
instruments designed to measure Eating Disorders, much is to be considered. Assessing an eating disorder
is a complex issue, as the Diagnostic and Statistical Manual (DSM-IV-TR)
defines three separate categories of eating disorders: Anorexia Nervosa,
Bulimia Nervosa, and Eating Disorders Not Otherwise Specified (NOS). The tests
I chose to compare are diametrically different. The widely-used and older test,
“Eating Disorder Inventory-3” (EDI-3) (Garner, Garfinkle, 1979) is based upon
20 years of research in eating disorders, and is in its third revision. It assesses
the eating disorder categories listed in the DSM, except Binge Eating Disorder.
However, “The EDI-3 is not designed to arrive at a diagnosis of eating
disorder. Instead the emphasis is placed on the measurement of psychological
traits relevant to the development and maintenance of such disorders” (Kagee,
1984-2004). The newer test, “Questionnaire For Eating Disorder Diagnosis”
(Q-EDD) (Mintz, O’Halloran, Mulholland, and Schneider, 1997) also
operationalizes eating disorder criteria of the DSM, but goes further than the
EDI-3, in that it differentiates: (a) between those with and without an eating
disorder diagnosis, (b) between symptomatic and asymptomatic individuals, and
(c) between anorexia and bulimia diagnosis. The Q-EDD also tested three
different groups, and included a supplementary clinical oral interview to
arrive at their research conclusions.
The EDI-3 can be administered in 20 minutes,
has 25 questions, and utilizes a Likert Scale. It can be used with individuals
or groups. The symptom checklist is written at a sixth grade level, thus it can
easily be scored by the administrator, and even a lay-person, such as a teacher
or athletic coach. This makes it an apt screening tool for academic and
athletic purposes, wherein the administrator could use the tests results for
referral. Again, “The rationale behind the development of the EDI-3 was to test
the continuum model of anorexia nervosa, which states that this disorder is the
final stage of a continual process beginning with voluntary dieting and
progressing to more stringent forms of dieting accompanied by progressive loss
of insight” (Kagee, 1984-2004). Although it has high cross-cultural validity
(both U.S. and international samples), group normative sample numbers were not
divulged, other than describing them as, “moderate-size samples of U.S. and
international male and female adults, as well as international male and female
adolescents” (Garner, 1984-2004).
Overall, its factor analysis validity is
poor, other than proving inverse convergent validity for low self-esteem scores
of .82 when compared to Rosenberg Self-Esteem Scale (Rosenburg, 1965) on a
nonclinical sample of 543 females. Exploratory factor analysis three-factor
model accounted for 63% of variance, and 60.8% and 65.6% of variance among
different sample groups (Atlas, 1984-2004).
Reliability was more impressive, ranging
from .90 to .97 across four diagnostic groups and three normative groups (Kagee,
1984-2004). Test-retest stability of scores of 34 females after 1-7 days: “Correlation
coefficients ranging from .86 (for Asceticism) to .98 (for Interpersonal
Alienation) suggested excellent stability of subscale and composite scores,
albeit on a very restricted study sample” (Atlas, 1984-2004). Overall, critics Kagee and Atlas (1984-2004) found
the EDD-3 to be disappointing in construct. Future utility of the instrument
seems to be for screening purposes only in detection of eating disorders, and
their progress on a spectrum.
The second test, “The Questionnaire for
Eating Disorder Diagnosis” (Q-EDD) (Garner, 1984-2004) “…represents the first
attempt in the field of eating disorders to provide a comprehensive assessment
of the widely used methodology of operationalizing the DSM into a questionnaire
format” (Mintz, O’Halloran, Mulholland, Schneider, 1997). It is a self-report,
50 questions, and takes 5 to 10 minutes to complete. The group sample was a
non-clinical group of 1,400 college women who completed three eating disorder
tests: the Q-EDD, revised Bulimia Test (BULIT-R; Thelen,
Farmer, Wonderlich, & Smith, 1991), and
the Eating Attitudes Test (EAT; Garner
& Garfinkel, 1979). In addition, participants completed a
structured interview by clinicians. I consider this to be a large sample group,
and rigorous construct methodology.
On Test 1, “Criterion validity was assessed
by an examination of the diagnosis yielded by the Q-EDD and those yielded by
clinical interviews, and accuracy rate was 98% and 90%. Incremental validity
was examined by comparing the level of agreement between Q-EDD diagnosis and
clinical interview diagnosis with the level of agreement between preexisting
inventory diagnosis and clinical interview diagnosis” (Mintz, O’Halloran,
Mulholland, Schneider, 1997). Incremental accuracy rates were 97% and 94%.
Hence, the Q-EDD and the BULIT-R were roughly equivalent in all aspects except
for positive predictive power: the Q-EDD was correct at predicting Bulimia 78%
of the time, whereas the BULIT-R was correct 54% of the time. The incremental
validity of the Q-EDD in comparison with the EAT was not examined, because
there was only one interview-defined anorexic. Test 1 test-retest was delayed
for 1-3 months, and scored 64% and 54%. This wider reliability score could be
due to waiting a longer period of time before retesting subjects. Inter-scorer
agreement was 100%, and fifty randomly selected Q-EDDs were scored by two
scorers.
On Test 2, 167 college women were tested,
and the test instruments were the same (Q-EDD, BULIT-R, and EAT). Convergent
validity could not be calculated, as there was only one bulimic. The
test-retest was calculated as follows: 94% and 85%; the higher scores would be
a result of retesting just 2 weeks later. Inter scorer agreement was 100%, and
50 randomly selected Q-EDDs were scored by two scorers.
On Study 3, “The purpose of Study 3 was to
assess criterion validity. Study 1 indicated that…the Q-EDD demonstrated good
criterion validity; we were thus interested in determining whether this good
criterion validity would hold for the clinical sample” (Mintz, O’Halloran,
Mulholland, Schneider, 1997). 37 participants were recruited by therapists, and
all had been diagnosed with eating disorders. In comparing the diagnosis of
clinicians to the Q-EDD, accuracy rates were 78%. The sensitivity and accuracy
rates for the differentiation of anorexia from bulimia were 100%, and the
false-negative rate was 0%.
Based on these three studies, “Strong
support was obtained for the psychometric properties of the Q-EDD…convergent
validity was demonstrated by significant correspondence between Q-EDD diagnosis
and scores on the BULIT-R and the EAT. Test-retest reliabilities found that
Q-EDD diagnosis were quite stable over a 2-week period and less stable over a 1
to 3 month period. The 100% inter-scorer agreement indicates that scoring of
the Q-EDD can be easily mastered” (Mintz, O’Halloran, Mulholland, Schneider,
1997). In terms of validity, due to the low number of anorexics in Study 1, the
Q-EDD and the EAT could not be compared. However, the high level of anorexia
diagnosis in Study 3 leads to the conclusion that Q-EDD is a better measure of
DSM anorexia than the EAT. The Q-EDD and the BULIT-R performed equally well on
measuring bulimia; therefore, clinicians wanting to distinguish bulimics from
nonbulimics could use either instrument. Although these conclusions build a
strong case for using the Q-EDD, perhaps the most significant psychometric
support was the criterion validity of the Q-EDD across both the clinical
interview and judgment scores. Accuracy rates were: 98% and 90% in Study 1, 78%
and 78% in Study 3. In differentiating anorexia from bulimia, accuracy rate was
100% in Study 3” (Mintz, O’Halloran, Mulholland, Schneider, 1997). Apparently,
the Q-EDD is very effective at differentiating a diagnosis of anorexia from
bulimia. Fairburn et al., 1990 and Williamson et al., 1995 wrote, “There is a
great need in the eating disorder field for an instrument that can
operationalize a full spectrum of eating disorders and make differential
diagnosis.” The Q-EDD may be the first questionnaire to achieve that goal. In
terms of future clinical utility, “Because the Q-EDD yields both a diagnosis
and frequency data for individual behaviors, it can be used to track progress
in therapy” (Mintz, O’Halloran, Mulholland, Schneider, 1997).
To use a crude analogy, one could compare
these two tests instruments as one would compare cars. They are both used for
driving. However, the EDI-3 would drive like the trusty family station wagon; a
time-tested and dependable ride, less concerned with safety, but roomy and easy
to drive. The Q-EDD would drive like a 10-year-younger protégé; designed with
safety, speed and utility in mind. If I had to pick from these two cars, I
wouldn’t hesitate to choose the Q-EDD, even though it’s a relative “newcomer”
to the market of testing instruments. It has greater validity, reliability,
test-retest reliability, and inter-rater agreement than the EDI-3. Although the
EDI-3 has served its purpose as an assessment tool, it’s time for a newer
construct which can accurately distinguish and diagnose eating disorders. Sometimes,
new is better.
References:
Garner, D. M., & Garfinkel, P. E. (1979). The
Eating Attitudes Test: An index of the symptoms
of anorexia nervosa. Psychological Medicine, 9,
273– 279.
Kagee, A.
(1984-2004). Review of the Eating
Disorder Inventory-3. Mental Measurements Yearbook and Tests in Print.
Accession Number: 17123228.
Mintz, L. B.,
O'Halloran, M., Mulholland, A. M., & Schneider, P. A. (1997). Questionnaire
for Eating Disorder Diagnoses: Reliability and validity of operationalizing
DSM—IV criteria into a self-report format. Journal of Counseling Psychology,
44(1), 63-79. doi:10.1037/0022-0167.44.1.63
Rosenberg, M. (1965). Society and the adolescent
self-image. Princeton, NJ: Princeton
University Press.
Atlas, J. A. (1984-2004).
Review of the Eating Disorder Inventory-3. Mental Measurements Yearbook and
Tests in Print. Accession Number: 17123228.
Thelen,
M. H., Farmer, J., Wonderlich, S., Smith, M. (1991). A revision of the Bulimia
Test: The BULIT—R. Psychological Assessment: A Journal
of Consulting and Clinical Psychology, Vol 3(1), pp. 119-124. US: American Psychological
Association.
Fairburn, C. G., Phil, M., & Beglin, S. J. (1990).
Studies of the epidemiology of bulimia
nervosa. American Journal of Psychiatry, 147,
401– 408.
Williamson, D. A., Anderson, D., Jackman, L. P., &
Jackson, S. R. (1995). Assessment of eating
disordered thoughts, feelings, and behaviors. In Allison
(Ed.), Handbook of assessment
methods for eating behaviors and weight-related
problems (pp. 303– 346). Thousand Oaks, CA:
Sage.
No comments:
Post a Comment