“Whatever else it produces, an organization is a factory that manufactures judgments and decisions. Every factory must have ways to ensure the quality of its products in the initial design, in fabrication, and in final inspections.”
Daniel Kahneman (2002, Nobel Prize in Economics; author of Thinking, Fast and Slow )
This blog entry is based on a talk entitled “Decision quality, problem-solving, and data science — Lean concepts for human-centered, data-informed decision-making”. Here are the slides:
Decision quality (DQ)
Decisions drive vision, strategy, execution, evaluation, problem-solving, performance, and continuous improvement. A decision is an irrevocable commitment of time and resources. Every decision has an opportunity cost—the loss benefit of the better option not chosen or not considered. Decision competent managers use and leverage data, information, and knowledge management systems to improve decision quality, problem-solving, and performance. Driven by decision makers’ priorities and questions, data scientists transform data into information and actionable knowledge to inform, influence, or optimize decision making.
Decision-making is the single most important activity we do every day. At the San Francisco Department of Public Health, Population Health Division, we promote decision quality (DQ) [2,3]. DQ starts by knowing what a good decision looks like. A good decision is built with six quality requirements (Table 1).
|Frame||Appropriate frame||What are we deciding and why?|
|Information||Relevant and reliable information||What do we need to know?|
|Choices||Creative, doable alternatives||What (evidence-based) choices do we have?|
|Reasoning||Sound reasoning and analysis||Are we thinking straight?|
|Commitment||Commitment to action||Is there commitment to action?|
|Prospects||Clear values and trade-offs||What consequences do we care about?|
The frame is the most important: defining the purpose, perspectives, and scope (key decisions). Prospects are the future states created by values, trade-offs, choices, actions, consequences, results, goals, impacts, and opportunity costs. Information is the data that has been analyzed and synthesized into actionable knowledge. This includes community voice, wisdom, and evidence. Choices represents creative, doable alternatives that include evidence-based practices. Reasoning is the analytical approach or methods used to select the best options. Commitment reminds us to have the right people participating in the decision-making, especially if they will be involved in implementing the decision(s).
Figure 1 depicts DQ as a deliberative process of divergence (information gathering, creative brainstorming) and convergence (criteria-based selection or prioritization).
SFDPH has adopted lean—systematically developing people to solve problems and consuming the fewest possible resources while continuously improving processes to provide value to community members and prosperity to society . The PHD also uses Results-Based Accountability for collective impact initiatives .
The daily management system (DMS) supports daily team problem-solving and leadership development. The DMS consist of five interdependent components:
- standard work (sets baseline for training and improvement),
- visual management (shared understanding/accountability),
- tiered reporting (strategic alignment and communication),
- team huddles (daily problem solving and improvement), and
- staff development (PDSA, A3 Thinking, leadership).
Problem-solving involves the PDSA steps:
- Plan: Problem definition (problem statement)
- Plan: Consequence (risk) analysis
- Plan: Root cause analysis
- Plan: Countermeasure design and testing
- Do: Countermeasure implementation
- Study: Countermeasure evaluation (intervention causal analysis)
- Act: Act on what you learn.
Figure 2 depicts the causal relationship between Problem, Consequences, Causes, and Countermeasures (Prevention, Control, Mitigation).
Decision making is embedded in every step (Figure 3), and the quality of each analysis is driven by decision quality.
The evaluation and care of patients illustrate these concepts. When a patient complains of “chest pain” (problem) the provider defines the problem in terms of possible immediate consequences (cardiac arrest from myocardial infarction vs. discomfort from esophageal reflux). The provider conducts a root cause analysis with a history and physical exam (primary data), selects a causal hypothesis, and selects diagnostic tests (information). Based on what the provider decides (hypothesizes) are the problem, consequences, and causes, the provider selects and designs countermeasures (treatments) appropriate for each.
Data, information, and knowledge management systems
Data, information, and knowledge management systems should be designed and deployed to support (inform, influence, or optimize) human decision making. Human-centered designed systems will anticipate humans’ needs, wants, or requests for data, information or knowledge. Data are collected and organized into data systems (e.g., chest X-ray). Information is data that has been analyzed and summarized (radiologist’s chest X-ray report). Knowledge is information from various sources that has been synthesized and prioritized for learning, awareness, sharing, reasoning, decision making, or action (e.g., scientific articles, clinical guidelines).
In public health and medicine we use and need both information systems and knowledge management systems. At SFDPH, Epic and Maven are information systems. “Knowledge management (KM) is the process of creating, sharing, using and managing the knowledge and information of an organization. It refers to a multidisciplinary approach to achieving organizational objectives by making the best use of knowledge.”1
Data, information, and knowledge will have flows (pull and push) and life cycles. Software and IT solutions enable, support, improve these human and business processes.
1. Kahneman D. Thinking, fast and slow. New York: Farrar, Straus; Giroux; 2011.
2. Aragón TJ, Colfax G. We will be the best at getting better! A playbook for population health improvement. UC Berkeley eScholarship [Internet]. 2019; Available from: https://escholarship.org/uc/item/9xg5t30s
3. Spetzler C, Winter H, Meyer J. Decision quality: Value creation from better business decisions. 1st ed. Wiley; 2016.
4. O’Donnell JP, Schroeder L. Public service: Lean’s next frontier? The Lean Post [Internet]. 2014; Available from: http://www.lean.org/LeanPost/Posting.cfm?LeanPostId=332
5. Friedman M. Trying hard is not good enough: How to produce measurable improvements for customers and communities, 10th anniversary edition. 3rd ed. Parse Publishing; 2015.