Using Non-experimental Data to Estimate Treatment Effects

By Elizabeth A. Stuart, PhD; Sue M. Marcus, PhD; Marcela V. Horvitz-Lennon, MD; Robert D. Gibbons, PhD; Sharon-Lise T. Normand, PhD; C. Hendricks Brown, PhD

Although much psychiatric research is based on randomized controlled trials (RCTs), where patients are randomly assigned to treatments, sometimes RCTs are not ethical nor feasible. Ethical concerns might preclude randomization, such as when evaluating whether "light cigarettes" produce less health risk by potentially randomizing subjects to smoke different brands, or it may be impractical, such as when the treatment of interest is widely available and commonly used. When RCTs are unethical or infeasible, a carefully constructed nonexperimental study can often be used to estimate treatment effects. Although nonexperimental studies are disadvantaged by lack of randomization, the study costs may be lower, the study sample may be broader, and follow-up may be longer, as compared to an RCT.

ABOUT THE AUTHORS

Elizabeth A. Stuart, PhD, is with Johns Hopkins Bloomberg School of Public Health, Baltimore. Sue M. Marcus, PhD, is with Mount Sinai School of Medicine, New York. Marcela V. Horvitz-Lennon MD, is with the Department of Psychiatry, University of Pittsburgh School of Medicine. Robert D. Gibbons, PhD, is with the Center for Health Statistics, University of Illinois at Chicago. Sharon-Lise T. Normand, PhD, is with the Department of Health Care Policy, Harvard Medical School, and Department of Biostatistics, Harvard School of Public Health, Boston. C. Hendricks Brown, PhD, is with the Department of Epidemiology and Public Health, University of Miami.

Address correspondence to: Elizabeth A. Stuart, PhD, Johns Hopkins Bloomberg School of Public Health, 624 N Broadway, Eighth Floor, Baltimore, MD, 21205; fax 410-955-9088; or e-mail estuart@jhsph.edu.

Dr. Stuart has disclosed the following relevant financial relationships: Center for Prevention and Early Intervention, jointly funded by the National Institute of Mental Health (NIMH) and the National Institute on Drug Abuse (Grant MH066247; PI: N. Ialongo), and NIMH grant K25-MH083846: research grant recipient. Dr. Marcus has disclosed no relevant financial relationships. Dr. Horvitz-Lennon has disclosed the following relevant financial relationships: research grant recipient (NIMH Grant P50-MH073469). Dr. Gibbons has disclosed the following relevant financial relationship: research grant recipient (NIMH Grant R56-MH078580 and R01-MH8012201). Dr. Normand has disclosed the following relevant financial relationship: research grant recipient (NIMH Grant MH61434). Dr. Brown has disclosed the following relevant financial relationship: research grant recipient (NIMH Grant R01-MH040859).

The authors are indebted to Larry Zaborski, MS, Harvard Medical School, for earlier programming help, and to Richard Frank, PhD, Harvard Medical School, for generously providing the Medicaid data. Dr. Zaborski and Dr. Frank have disclosed no relevant financial relationships.

doi: 10.3928/00485713-20090625-07

EDUCATIONAL OBJECTIVES

1. Define the strengths and weaknesses of observational studies.

2. List methods for the identification and control of bias in observational studies.

3. Recognize propensity score matching methods.

 



Design and Analysis of Longitudinal Studies, Part 2


Of the Terrible Doubt of Appearances
Jan Fawcett, MD

Design and Analysis of Longitudinal Studies, Part 2
Robert D. Gibbons, PhD

A 54-year-old Man with History of PTSD
Violeta O. Tan, MD; Natalie L. Rasgon, PhD

 Where Do We Go Wrong in Assessing Risk Factors, Diagnostic and Prognostic Tests? The Problems of Two-by-two Association
Helena Chmura Kraemer, PhD; Robert D. Gibbons, PhD

 Statistical Approaches to Modeling Multiple Outcomes in Psychiatric Studies
Armando Teixeira-Pinto, PhD; Juned Siddique, DrPH; Robert D. Gibbons, PhD; Sharon-Lise T. Normand, PhD

 Why Does the Randomized Clinical Trial Methodology So Often Mislead Clinical Decision Making? Focus on Moderators and Mediators of Treatment
Helena Chmura Kraemer, PhD; Robert D. Gibbons, PhD

The Role of ANCOVA in Analyzing Experimental Data
Thomas R. Belin, PhD; Sharon-Lise T. Normand, PhD