Some Principles of Assessment and Some Ethical Considerations for the Evaluating Psychologist

By Sally Singer Horwatt, Ph.D

 

To the question, why do we do presurgical bariatric evaluations we ask, in the manner of the Talmudic scholar, why shouldn’t we do bariatric evaluations? These are very good questions. Consider these questions: Why do the seriously obese people require a psychological evaluation, but not those who go in for cosmetic surgery or elective lumbar surgery? Second, years of research have failed to provide consistent evidence that psychopathology has any consistent, well-documented association with weight loss. So what are we looking for in these evaluations? How many psychologists even get a referral question other than “Will you do a pre-surg eval?” from the surgeon? What are the criteria? What are the definitions of a good outcome?

Ashton, et al (2008) make a noteworthy point: “If psychological testing is being used as a form of preoperative triage, both patients and surgeons are entitled to know whether there is sufficient evidence to justify its use in this way…”

If a patient is being denied a potentially life-saving procedure on the basis of psychological tests which have no predictive value and goes on to suffer the effects of a preventable comorbidity (diabetes, myocardial infarction, etc.) then not only the patient, but presumably, their legal representative, will be entitled to ask some searching questions concerning the rationale behind such a decision.

But to answer the original question, why do we do presurgical bariatric evaluations, there is a practical reason. First, when the fact of the growing number of seriously obese people is coupled with the fact that surgery is the most efficacious and longer lasting method than other weight loss methods then, predictably, more people are electing the surgery. The growing number of surgeries, understandably, has already begotten the machinery for best practices guidelines and for standards of care. For those reasons, health insurers and the ASM&BS require a presurgical evaluation. The surgeons and health insurers have evolving standards of care which requires them to get their patients presurgical psychological evaluations. So, we do them…for the “integrated health team” and for the patient who might, otherwise, be denied treatment.

However, as most psychologists who do assessments have been taught, the goal is not merely to describe the person, but to develop relevant answers to specific questions, aid in problem-solving, and facilitate decision-making (Groth-Marnat, 2009). Stickle and Weems (2006) have noted that “…much of what is typically attained from our assessment tools and procedures is more accurately termed “enlightenment”…we are often generally “enlightened” by the results of clinical assessments, but have limited ability to predict specific behaviors across circumstances.”

In a chapter in Bootzin and McKnight’s (2006), Smith describes “rear end validity”. He notes that the most meaningful and enduring methodological terms used today were named during the decade of the 50’s: Cronbach and Meehl and construct validity, Meehl and Rosen operationalized the concepts of specificity, sensitivity and efficacy of cutting scores; Campbell and Fisk conceptualized the usefulness of the multimethod-multitrait matrix and added convergent validity and discriminant validity. Campbell and Stanley elaborated on the concept of quasi-experimentation and defined the terms internal validity and external validity. To these he adds, “rear-end validity” which he describes as the habit of relying on tradition or accepted practices, rather than sound scientific thinking, to guide procedural decisions in the behavioral sciences.

Smith attributes the “malignant growth” within measurement batteries as examples of rear end validity the antidote for which is Sechrest’s concept of incremental validity. That is, “once a key measure is selected, it is incumbent on any additional measure to make a conceptually and statistically significant contribution to understanding the construct of interest.” Complicating the matter, according to (Wonderlich, 2002), most empirical data suggests that personality traits are continuously distributed as dimensions rather than bimodal categories, cutoff points for “diagnoses” are arbitrary, show poor discriminant validity between categories, and do not show stability over time.

On the other hand, traditional self-report measures have a number of limitations insofar as they require participants to report on behaviors, beliefs or feelings that occurred hours, days, weeks or even years prior to their responding. Memory bias research shows that recall of attitudes or emotions is consistently influenced by current attitudes or emotions. Also, when participants are asked to recall information about behavior or mood, the most recent or salient events will influence ratings of behavior. (Engel, S.D., et al, 2005).

According to Rutledge et al, (2010),

There is a lack of compelling statistical evidence to support that these assessments are accomplishing their intended goals. Across at least five review of the pre-bariatric psychological assessment literature published in the past decade, none suggest that psychosocial factors consistently predict weight-related surgery outcomes.

Two suggestions about improving prediction are: limiting our scope to bounded predictions, using a test or measure to make an inference about a person that is limited in time, situation or range of behavior (Stickle, 2006). Second, an effective means of achieving predictive information might be simply to ask the patients to predict their own behaviors. In some circumstances, self-prediction has been found to be more accurate than psychological tests (Groth-Marnat, 2009).

Research has not identified consistent contraindications to bariatric surgery or psychosocial indicators of poor outcome. According to Fabricatore (2006), unconditional approval is the most common outcome. He questions whether recommendations to postpone or forego surgery are potentially harmful. In other words, is it ethical to recommend against surgery in the absence of data-based contraindications. The most immediate task facing researchers is to identify preoperative psychiatric, behavioral, cognitive and environmental factors that are shown to be related to treatment failure (assuming a consensus on “treatment failure”), serious postoperative complications or death. The Longitudinal Assessment of Bariatric Surgery study, an ongoing multisite investigation funded by the NIH, should soon provide such data.

Finally, and a very important issue, is examiner confirmation bias. The research on obesity is extensive. Several dynamic, behavioral and cognitive/behavioral hypotheses have been tested. Yet, with their hypotheses unconfirmed, many of the papers’ discussion sections conclude with statements calling for further research until the correlation has been confirmed. For example, “Even where psychosocial functioning does not predict outcome, it is important to identify the patients’ characteristics which make be linked to their prognosis and to provide necessary pre- and postoperative psychosocial intervention” (Gerbrand, 2005).

Or if they find that depression, OCD, or a personality disorder is associated with obesity, the “pathology” must be resolved before the surgery. However, patients who are delayed from bariatric surgery have not been shown to have improved weight loss (Madan, et al, 2008).

Additionally, Alverdy, et al (2009) point out,

The currently held notion that patients who fail to lose weight after bariatric surgery are “non-compliant” with the dietary restriction imposed by the surgery, whereas those that succeed are universally compliant because of personal commitment is scientifically baseless. The complex interaction between the environment and genetics of both the host and its microbial flora must be tracked over the course of weight loss …before any meaningful conclusion can be made regarding the mechanisms of action of a given operative intervention and its success or failure.

In a survey of U.S. bariatric surgeons using three hypothetical patients, with a 62.5% response rate, hypothetical patient age, BMI, and social support were the most influential variables determining physician selection. Patient race did not play a role in surgeons’ decisions to operate. Overall, younger age, older age, limited functional status, poor social support, self-pay and public insurance were associated with decreased odds of selection. BMI and comorbidity criteria influenced the magnitude of these effects. (Santry, HP et al, 2007.) So, it seems, many surgeons seem to rely on demographic variables.

Healthcare proponents of bariatric surgery acknowledge that surgery will not remain the answer to obesity.

Weight loss surgery is the only sustainable weight loss option for the majority of adults. As long as the person is motivated to make the lifestyle changes required by the various weight loss procedures, he or she can enjoy the benefits of less comorbidity, fewer medications, lower costs, and an improved quality of life. In the long term, however, surgery will not be the answer…Not enough surgeons are available to address the current obese adults population, and the prevalence of obesity continues to rise…There is no quick fix.

Summary: Healthcare providers choose to do presurgical evaluations because not doing them would deprive a patient a potentially life-saving treatment which not only helps them lose weight, but improves or cures serious comorbidities, even if the patient regains some of the lost weight. They might choose not to do so because they are expensive, time consuming, and cost the patient a great deal of money. As important, they have little criterion validity or predictive utility, and are usually sources of confirmation bias. Finally, many surgeons make their own judgments, regardless of the putative psychopathology of the patient.

 
References

 
1. Ashton, D; Favretti, F; Segato, G. (2008). Preoperative Psychological Testing—Another Form of Prejudice. Obes Surg. Oct;18 (10):1330-7.

2. Alverdy, J; Prachand, V; Flanagan, B; Thislethwaite, W; Siegler, M; Garfinkel, M; Angelos, P; Agarwal, S; Santry, H. (2009). J. Gastrointest.Surg. Mar;13(3):465-77.

3. Bootzin, R.R., & McKnight, P.E. (Eds., 2006). Strengthening Research Methodology: Psychological Measurement and Evaluation. Washington, D.C.: APA Books.

4. Engle, SG; Wonderlich, SA; & Crosby, RD. (2005). Ecological Momentary Assessment in Assessment of Eating Disorders. The Guilford Press, New York.

5. Fabricatore, A; Crerand, C; Wadden, T; Sarwer, D; Krasucki, J. (2006). How Do Mental Health Professionals Evaluate Candidates for Bariatric Surgery? Survey Results. Obesity Surgery, 16, 567-573.

6. Gerbrand, C. (2005) Psychosocial Predictors of Success Following Bariatric Surgery. Obesity Surgery, Volume 15, Number 4, 552-560.

7. Groth-Marnat, G.(2009) Handbook of Psychological Assessment. 5th Edition. John Wiley & Sons: Hoboken, New Jersey.

8. Khaitan, L and Smith, D. (2005) Obesity in the United States: Is There A Quick Fix? Pros and Cons of Bariatric Surgery from the Adult Perspective. Current Gastroenterology Reports, 7: 451-454.

9. Madan, A; Dhawan, D; Coday, Mace; Tichansky, D. (2008). Patients Who are Delayed from Undergoing Bariatric Surgery Do Not Have Improved Weight Loss. Obesity Surgery, Volume 18, Number 3, Pages 278-281.

10. Rutledge, T; Adler, S; Friedman, R. (2010). A Prospective Assessment of Psychosocial Factors Among Bariatric Versus Non-Bariatric Surgery Candidates. Obesity Surgery, September 25, 2010.

11. Santry, H; Lauderdale, D; Cagney, K; Rathouz, P; Alverdy, J; and Chin, M, (2007) Predictors of Patient Selection in Bariatric Surgery. Annals of Surgery, January; 245 (1): 59–67.

12. Stickle, T. & Weems, C. (2006). Improving Prediction from Clinical Assessment: The Roles of Measurement, Psychometric Theory and Decision Theory. In R.R. Bootzin & P.E. McKnight, Strengthening Research Methodology, Psychological Measurement and Evaluation, 213-230. Washington, D.C. American Psychological Association.

13. James E Mitchell, Kathryn L Lancaster, Melissa A Burgard, L Michael Howell, Dean D Krahn, Ross D Crosby, Stephen A Wonderlich and Blake A Gosnell (2001). Long-term Follow-up of Patients’ Status after Gastric Bypass. Obes Surg. 2001 Aug; 11(4):464-8.

14. Wonderlich, S. A. (2002). Personality and eating disorders. In Fairburn, C.G. & Brownell, K.D. (eds.) Eating disorders and obesity, 2nd ed. A comprehensive handbook. (pp 50-61). New York: The Guilford Press.

 
Top of Page