A Quarter Century of Multilevel Models in Social Epidemiology

A Short Story in Three Acts

Sam Harper

2023-06-14

  • Multilevel methods developed in 1980s in demography (Entwistle), statistics (Wong/Mason), education (Bryk/Raudenbush).
  • Critical need for theory.

The development of multi-level approaches in epidemiologic research may facilitate research which elucidates the independent and joint effects of individual and environmental factors on health behaviors and health outcomes.

Epidemiology has lost its way

Social context and ‘population perspective’ has been forgotten.


Multilevel health determinants

Social context a crucial element of conceptual models for ‘social determinants of health’


Why multilevel social epidemiology?

  • Place-based comparisons of health are revealing (Villermé, Farr, Graunt, Snow, DuBois, many others)

  • Communities inherently reflect social dynamics.

  • Host-Agent-Environment (physical and social).

  • “Population perspective”, contra biomedical individualism.

John Snow’s ‘Grand Experiment’

Act 1: The Big Idea

Early influential studies in social epidemiology


Neighborhood ‘effects’ on violence, mortality

  • Focus on mutual adjustment
  • Clustering addressed as nuisance

Extended to CVD, low birthweight, other outcomes

  • Random effects implementation
  • Exploration of multi-level EMM

  • Strong theory, field measurements, sophisticated models, potential mechanisms linked to violent crime.

Multilevel analyses showed that a measure of collective efficacy yields a high between-neighborhood reliability and is negatively associated with variations in violence, when individual-level characteristics, measurement error, and prior violence are controlled. Associations of concentrated disadvantage and residential instability with violence are largely mediated by collective efficacy.

Focus on ‘simultaneous’ effects:

By incorporating multiple levels of determination in the study of individual outcomes, multilevel analysis allows for the effects of macro- and micro-level variables as well as their interactions

Potential:

Multilevel analysis is one way to begin to restore a population or societal dimension to epidemiologic research

The ‘Big Idea’:

The big idea is that what matters in determining mortality and health in a society is less the overall wealth of that society and more how evenly wealth is distributed.

  • Inequality = contextual, but how?

State of the Evidence: 2001

  • 25 studies but only 10 used multilevel models, however…

In 23 of the 25 studies we identified, researchers reported a statistically significant association between at least one neighbourhood measure of socioeconomic status and health, controlling for individual socioeconomic status.

  • Potential for intervention:

…serve the purpose of identifying types of geographical areas where traditional public health interventions, aimed at individual risk reduction, may best be targeted.

Traditional measures of association such as odds ratios thus provide an incomplete epidemiological basis for decision making in public health interventions.

Act 2: A Crisis of Confidence?

Large-scale ‘multilevel’ RCT

  • ~4600 families in high poverty randomized to housing vouchers.

  • Generated large differences in exposure to high-poverty neighborhoods.

  • 5-year follow-up (2003):

    • No impacts on economic self-sufficiency of mothers.
    • Other outcomes mixed, some positive, some negative.
  • Many limitations.

A skeptical view

The recent and enthusiastic adoption of the multilevel model for neighborhood effects research appears to be a case of statisticism, a term used to describe an almost ritualistic appeal to significance testing and both sampling and measurement error when they are not the problem

What are the problems?

  • Social stratification
  • Endogeneity
  • Extrapolation
  • Spillovers

Income inequality: not so bad for health?


  • Evidence for the income inequality/health link was “slowly dissipating”

  • Multilevel studies inconsistent in US.

  • Weak evidence from Europe and Asia.

  • Individual-level controls matter.

Fixed effects: No. 

Random effects: Yes!

Neighborhood evidence to 2007

  • 86 multilevel papers on neighborhoods
  • 80% cross-sectional designs
  • Inconsistencies within and across studies.

Neighborhood effects at 20 years

“it is not clear how much we are learning, or whether such lessons are improving population health…experimental evidence of neighborhood effects is mixed, and observational studies too often report mere correlations, side-stepping critical effect identification issues. Since epidemiologists have long known that disadvantaged environments are not healthy, the utility of studies that do not face the difficult methodological challenges is questionable”

Act 3: A Way Forward?

Merging of multilevel and causal inference

  • Greater focus on credible study designs.

    • Cluster RCTs
    • Quasi-experiments
  • Utilizing longitudinal data to focus on changes in exposure

  • Weighting methods to deal with observables and post-exposure covariates

  • Extensions to mediation


All fit within the scope of multilevel design and analysis

Methods development and clarification

  • Defining assumptions for causal effects of contextual exposures

  • Accounting time-varying exposures and confounding in a multilevel context

  • Conditional vs. marginal effects

Hong and Raudenbush (2008)

Nandi and Kawachi (2011)

Healthy discussion of MTO design / results

Observational data as a neighborhood experiment

  • Time-varying covariates controlled using IPTW, exposure effects estimated using MSMs.

  • Can replicate MTO findings.

  • Found significant lagged effect of living in concentrated disadvantage compared with advantage at wave 1

  • Recent review of ‘causal analyses’ of neighborhood effects.

  • Much more mixed.

  • Evidence of selection and confounding.

  • Lots of heterogeneity.

  • Stronger evidence for children than adults.

What about community RCTs?

Summary: What have we learned?


Multilevel models

  • Helped to push social epi forward.
  • Perhaps short of promises.
  • More cross-sectional random effects designs unlikely to help.

Neighborhood effects

  • Heterogeneous but reliably negative associations between adverse neighborhood conditions and health.
  • Particularly for children with longer exposure.
  • Potential underutilization of cluster-randomized interventions.

References

Anderson RT, Sorlie P, Backlund E, Johnson N, Kaplan GA. Mortality Effects of Community Socioeconomic Status: Epidemiology. 1997 Jan;8(1):42–7.
Cerdá M, Diez-Roux AV, Tchetgen Tchetgen E, Gordon-Larsen P, Kiefe C. The Relationship Between Neighborhood Poverty and Alcohol Use: Estimation by Marginal Structural Models. Epidemiology. 2010 Jul;21(4):482–9.
Chyn E, Katz LF. Neighborhoods Matter: Assessing the Evidence for Place Effects. NBER Working Paper. 2023;
Congdon P. Multilevel and Clustering Analysis of Health Outcomes in Small Areas. European Journal of Population. 1997;13(4):305–38.
Dahlgren G, Whitehead M. Policies and strategies to promote social equity in health. Stockholm, Sweden: Institute for Future; 1991.
Diderichsen F, Hallqvist J. Social inequalities in health: Some methodological considerations for the study of social position and social context. Inequality in healtha Swedish perspective Stockholm: Swedish Council for Social Research. 1998;25–39.
Diez-Roux AV. Bringing context back into epidemiology: Variables and fallacies in multilevel analysis. American Journal of Public Health. 1998 Feb;88(2):216–22.
Diez-Roux AV, Nieto FJ, Muntaner C, Tyroler HA, Comstock GW, Shahar E, et al. Neighborhood Environments and Coronary Heart Disease: A Multilevel Analysis. American Journal of Epidemiology. 1997 Jul;146(1):48–63.
Ecob R. A Multilevel Modelling Approach to Examining the Effects of Area of Residence on Health and Functioning. Journal of the Royal Statistical Society Series A (Statistics in Society) [Internet]. 1996 [cited 2023 Jun 13];159(1):61. Available from: https://www.jstor.org/stable/10.2307/2983469
Editor’s choice. BMJ. 1996 Apr;312(7037):0–0.
Galster G, Sharkey P. Spatial Foundations of Inequality: A Conceptual Model and Empirical Overview. RSF: The Russell Sage Foundation Journal of the Social Sciences. 2017 Feb;3(2):1–33.
Glymour MM, Mujahid M, Wu Q, White K, Tchetgen Tchetgen EJ. Neighborhood Disadvantage and Self-Assessed Health, Disability, and Depressive Symptoms: Longitudinal Results From the Health and Retirement Study. Annals of Epidemiology. 2010 Nov;20(11):856–61.
Hong G, Raudenbush SW. Causal Inference for Time-Varying Instructional Treatments. Journal of Educational and Behavioral Statistics. 2008 Sep;33(3):333–62.
Hubbard AE, Ahern J, Fleischer NL, Laan MVD, Lippman SA, Jewell N, et al. To GEE or Not to GEE: Comparing Population Average and Mixed Models for Estimating the Associations Between Neighborhood Risk Factors and Health. Epidemiology. 2010 Jul;21(4):467–74.
Kaplan GA, Pamuk ER, Lynch JW, Cohen RD, Balfour JL. Inequality in income and mortality in the United States: Analysis of mortality and potential pathways. BMJ. 1996 Apr;312(7037):999–1003.
Kennedy BP, Kawachi I, Prothrow-Stith D. Income distribution and mortality: Cross sectional ecological study of the Robin Hood index in the United States. BMJ. 1996 Apr;312(7037):1004–7.
Krieger N. Epidemiology and the web of causation: Has anyone seen the spider? Social Science & Medicine. 1994 Oct;39(7):887–903.
Lynch J, Smith GD, Harper S, Hillemeier M, Ross N, Kaplan GA, et al. Is income inequality a determinant of population health? Part 1. A systematic review. The Milbank Quarterly. 2004;82(1):5–99.
Mackenbach JP. Income inequality and population health. BMJ. 2002 Jan;324(7328):1–2.
Mellor JM, Milyo J. Is Exposure to Income Inequality a Public Health Concern? Lagged Effects of Income Inequality on Individual and Population Health. Health Services Research. 2003 Feb;38(1p1):137–51.
Merlo J. Multilevel analytical approaches in social epidemiology: Measures of health variation compared with traditional measures of association. Journal of Epidemiology & Community Health. 2003 Aug;57(8):550–2.
Merlo J. A brief conceptual tutorial of multilevel analysis in social epidemiology: Linking the statistical concept of clustering to the idea of contextual phenomenon. Journal of Epidemiology & Community Health. 2005 Jun a;59(6):443–9.
Merlo J. A brief conceptual tutorial on multilevel analysis in social epidemiology: Interpreting neighbourhood differences and the effect of neighbourhood characteristics on individual health. Journal of Epidemiology & Community Health. 2005 Dec b;59(12):1022–9.
Merlo J. A brief conceptual tutorial of multilevel analysis in social epidemiology: Using measures of clustering in multilevel logistic regression to investigate contextual phenomena. Journal of Epidemiology & Community Health. 2006 Apr;60(4):290–7.
Moyer R, MacDonald JM, Ridgeway G, Branas CC. Effect of Remediating Blighted Vacant Land on Shootings: A Citywide Cluster Randomized Trial. American Journal of Public Health. 2019 Jan;109(1):140–4.
Nandi A, Kawachi I. Neighborhood Effects on Mortality. In: Rogers RG, Crimmins EM, editors. International Handbook of Adult Mortality. Dordrecht: Springer Netherlands; 2011. p. 413–39.
O’Campo P, Gielen AC, Faden RR, Xue X, Kass N, Wang MC. Violence by male partners against women during the childbearing year: A contextual analysis. American Journal of Public Health. 1995 Aug;85(8_Pt_1):1092–7.
O’Campo P, Xue X, Wang MC, Caughy M. Neighborhood risk factors for low birthweight in Baltimore: A multilevel analysis. American Journal of Public Health. 1997 Jul;87(7):1113–8.
Oakes JM. The (mis)estimation of neighborhood effects: Causal inference for a practicable social epidemiology. Social Science & Medicine. 2004 May;58(10):1929–52.
Oakes JM, Andrade KE, Biyoow IM, Cowan LT. Twenty Years of Neighborhood Effect Research: An Assessment. Current Epidemiology Reports. 2015 Mar;2(1):80–7.
Pearce N. Traditional epidemiology, modern epidemiology, and public health. American Journal of Public Health. 1996 May;86(5):678–83.
Pickett KE. Multilevel analyses of neighbourhood socioeconomic context and health outcomes: A critical review. Journal of Epidemiology & Community Health. 2001 Feb;55(2):111–22.
Riva M, Gauvin L, Barnett TA. Toward the next generation of research into small area effects on health: A synthesis of multilevel investigations published since July 1998. Journal of Epidemiology & Community Health. 2007 Oct;61(10):853–61.
Sampson RJ, Raudenbush SW, Earls F. Neighborhoods and Violent Crime: A Multilevel Study of Collective Efficacy. Science. 1997;277(5328):918–24.
Sampson RJ, Sharkey P, Raudenbush SW. Durable effects of concentrated disadvantage on verbal ability among African-American children. Proceedings of the National Academy of Sciences. 2008 Jan;105(3):845–52.
Schwartz S. The fallacy of the ecological fallacy: The potential misuse of a concept and the consequences. American Journal of Public Health. 1994 May;84(5):819–24.
South EC, MacDonald JM, Tam VW, Ridgeway G, Branas CC. Effect of Abandoned Housing Interventions on Gun Violence, Perceptions of Safety, and Substance Use in Black Neighborhoods: A Citywide Cluster Randomized Trial. JAMA Internal Medicine. 2023 Jan;183(1):31–9.
Subramanian SV, Blakely T, Kawachi I. Income Inequality as a Public Health Concern: Where Do We Stand? Commentary on "Is Exposure to Income Inequality a Public Health Concern?". Health Services Research. 2003 Feb;38(1p1):153–67.
Subramanian SV, O’Malley AJ. Modeling Neighborhood Effects: The Futility of Comparing Mixed and Marginal Approaches. Epidemiology. 2010 Jul;21(4):475–8.
Susser M, Susser E. Choosing a future for epidemiology: II. From black box to Chinese boxes and eco-epidemiology. American Journal of Public Health. 1996 May;86(5):674–7.
Von Korff M, Koepsell T, Curry S, Diehr P. Multi-level Analysis in Epidemiologic Research on Health Behaviors and Outcomes. American Journal of Epidemiology. 1992 May;135(10):1077–82.