Program theory is for the DAGs! (Part 1)

Every public health intervention has a program theory; however, very few can actually describe the program theory supporting their primary programmatic activity or research. Can you? If not, read on.

I too could not describe the program theories supporting my own work until I read Funnell Rogers’ book Purposeful Program Theory [1]. It turns out that program evaluators not only live and breath program theory, but they call it by different names: logic model, program logic, theory of change, causal model, results chain, intervention logic, etc. Hence the confusion!

From BetterEvaluation.org: “A program theory explains how an intervention (a project, a program, a policy, a strategy) is understood to contribute to a chain of results that produce the intended or actual impacts.”

First, quickly review how program evaluators describe program theory from BetterEvaluation.org:

http://www.betterevaluation.org/en/plan/define/develop_logic_model

Second, quickly review an example of program theory from BetterEvaluation.org (PDF document):

http://www.betterevaluation.org/sites/default/files/Define%20-%20Compact.pdf

The public health intervention depicted in the PDF document (link above) is delivering apples to schools to improve health. Figure 1 depicts this intervention as an “outcome hierarchy” (very similar to a driver diagram).

Alternatively, Figure 2 depicts this intervention as a “results chain.” The results chain is the historical precursor of the logic model.

In public health, the logic model is very popular. For me, a logic model is a good high-level summary for non-technical purposes (summary, communication, etc.); however, I do not like them (or variants above) as a place to start. For me, program theory is for the DAGs—directed acyclic graphs—and must include the theories of causation, change, and action.

Program theory for population heatlh

Program theory has three components and answers why? what? and how?:

• theory of causation (Why? primary causal links before an intervention),
• theory of change (What? key strategies to activate change), and
• theory of action (How? specific interventions and activities).

Theory of causation (Why? root causes)

What are the primary causal links of concern before implementing any intervention? Figure 3 depicts the theory of causation.

Figure 3: Theory of causation

This stage of program theory is the most important because nothing else matters if you get this wrong. In performance (quality) improvement we call this stage root cause analysis.

Theory of change (What? best strategy)

What primary strategy will we use to activate favorable change? For this example the strategy was to provide a nutrient item that will activate the three causes (rich in Vitamin C and quercitin, and with no added sugar). After further research eat whole red apples was selected as the primary strategy (“An apple a day keeps the doctor away.”). Figure 4 depicts the theory of action.

Figure 4: Theory of change (strategy to acivate favorable change)

Theory of action (How? best intervention)

What specific intervention will we deployed and how? After further research delivery of red apples to schools was selected as the primary intervention. Figure 5 depicts the theory of change.

Figure 5: Theory of action (specific intervention and activities to deploy primary strategy)

Final thoughts

Many times the theory of action is subsumed into the “theory of change,” especially when (a) the strategy and the intervention (action) are essentially the same, or (b) the strategy is a collection of reinforcing actions. In other words, a DAG may not explicitly have a node (or nodes) explicitly labeled as strategy, but strategy is represented by a collection of action nodes.

Sometimes the theory of causation is subsumed into the theory of change, especially when the underlying cause is well established and understood. Unfortunately, theory of causation is actually most often forgotten! Strategies and interventions are deployed without clarity of the underlying causal assumptions. Most public health staff are not trained in root cause analysis, causality, causal inference, or causal graphs (directed acyclic graphs) [24].

Many program evaluators only use the term “theory of change.” When you hear “theory of change” think “program theory”. I prefer (and highly recommend) embracing progam theory as the theories of causation (root causes), change (strategies), and action (interventions) which are distinct but connected. Use these three levels to tell a logical, coherent story. Use DAGs to build your story with clarity and causal rigor.

Without good program theory you cannot adequately

• explain key programmatic activities;
• evaluate key programmatic activities;
• design to test new interventions,
• design data collection, and
• analyze data for valid inferences.

A logic model is a good high level summary—but not a substitute!—for good program theory using DAGs. Logic models are good for communicating to non-technical audiences, but lack causal rigor for sound reasoning and technical work.

Conclusion: Every high-priority, high-stakes, or high-cost public health intervention, evaluation, or research must have an agree-uponed program theory, preferably using DAGs.

To read Part 2 go here.

Appendix: R code

Here is the R markdown for producing the DAGs above. You must install the DiagrammeR package in R/Rstudio.

Here is the R markdown for producing Theory of Causation figure (Figure 3):

{r tocause, echo=FALSE, fig.cap="Theory of causation", fig.height=2.5}
library(DiagrammeR)
grViz("
digraph tocause {
# Graph
graph [rankdir=LR, overlap=true]

# Nodes
node [fontname=Helvetica, fontsize=10, style=filled, fillcolor=LightCyan]
V [label='Vitamin C level']
Q [label='Quercitin level']
B [label='Body Mass Index']
H [label='Health']

# Edges
V->H Q->H B->H
}
")


Here is the R markdown for producing Theory of Change figure (Figure 4):

{r tochange, echo=FALSE, fig.height=2.5, fig.cap="Theory of change (strategy to acivate favorable change)"}
library(DiagrammeR)
grViz("
digraph tochange {
# Graph
graph [rankdir=LR, overlap=true]

# Nodes
node [fontname=Helvetica, fontsize=10, style=filled, fillcolor=MistyRose]
N [label='Eat whole\n red apples']
node [fontname=Helvetica, fontsize=10, style=filled, fillcolor=LightCyan]
V [label='Vitamin C level']
Q [label='Quercitin level']
B [label='Body Mass Index']
H [label='Health']

# Edges
N->V; N->Q; N->B
V->H; Q->H; B->H
}
")


Here is the R markdown for producing Theory of Action figure (Figure 5):

{r toaction, echo=FALSE, fig.height=3, fig.cap="Theory of action (specific intervention and activities to deploy primary strategy)"}
library(DiagrammeR)
grViz("
digraph toaction {

# Graph
graph [rankdir=LR, overlap=true]

# Nodes
node [fontname=Helvetica, fontsize=10, style=filled, fillcolor=Honeydew]
D [label='Delivery of\n red apples\n to schools']
node [fontname=Helvetica, fontsize=10, style=filled, fillcolor=MistyRose]
N [label='Eat whole\n red apples']
node [fontname=Helvetica, fontsize=10, style=filled, fillcolor=LightCyan]
V [label='Vitamin C level']
Q [label='Quercitin level']
B [label='Body Mass Index']
H [label='Health']

# Edges
D->N;
N->V N->Q N->B;
V->H; Q->H; B->H;
}
")


Option: To display R code change echo = FALSE to echo = TRUE.

References

1. Funnell SC, Rogers PJ. Purposeful program theory: Effective use of theories of change and logic models. San Francisco, CA: Jossey-Bass; 2011.

2. Fenton N, Neil M. Risk assessment and decision analysis with bayesian networks. CRC Press; 2012.

3. Scutari M, Denis J-B. Bayesian networks: With examples in r. Boca Raton: Chapman; Hall; 2014.

4. Pearl J, Glymour M, Jewell NP. Causal inference in statistics: A primer. 1st ed. Wiley; 2016.

Tomás J. Aragón
Health Officer, City & County of San Francisco; Director, Population Health Division

I exercise legal authority to protect and promote equity and health, and I direct core public health services.