Clinical Informatics



utility vs value utility is a function of value and also risk aversion, personal preferences / circumstances

Availability bias overestimating the probability of unusual events because of recent or memorable instances
Omission bias tendency to avoid interventions that may have serious side effects (vs comission bias - inaction felt to be less desirable)
Visceral bias tendency for emotions to influence our decisions
Confirmation bias tendency to corroborate rather than refute
Outcome bias tendency to judge decisions based on past outcome rather than the quality of the decision
Representativeness heuristic (bias) overestimate rare diseases by matching patients to 'typical picture' of that disease.
Anchoring Failure to adjust probability of disease/outcome based on new information, akin to 'premature closure'
Value-induced bias Overestimate probability of an outcome based on value associated with that outcome.
Conjunction fallacy assumed that specific conditions are more probable than a single general one (linda is a bank teller and feminist > linda is a bank teller)

PROACTIVE framework PROblems and objectives (define the Problem, Reframe from different perspectives, identify Objectives for action); consider Alternatives (one of wait-and-see, initiate intervention, obtain more information), model the Consequences, estimate the value of Trade-offs; Integrate the evidence and optimize Values, Explore assumptions and evaluate uncertainty
Markov Decision Process MDP a discrete time stochastic control process. Decision making framework for situations where outcomes are partly random and partly under control of decision maker; useful for progression of chronic disease and simulation with Monte Carlo methods
Heuristic Decision Making process of making decisions based upon using simple and efficient rules
Blois' funnel Breadth of diagnostic considerations are refined, restricted over course of interaction between patient and practitioner

Decision Science provides structure and guidance for systematic thinking; based on logical principles, informed by what we know about the limitations of human judgment and decision-making; allows logical and consistent analysis of complex decisions

Decision analysis the formal application of philosophy, methodology, and professional practice to understand and aid in decision-making
Influence diagram ID nodes: decision-rectangle, uncertainty-oval (RVs), value-octagon/diamondl & arcs (functional, conditional, informational); emphasizes probabilitstic relationships among variables; an ID composed only of chance nodes is a Bayseian belief network
Decision tree Decision node (decision maker in control) is square, chance node (probabilities) is circle, outcome node is triangle
What if / Sensitivity Analysis Use a range of values at the nodes to see how the model changes. Plot Expected value vs estimated probability. Lines intersect when standard of care = med x

Bayes theorem P(A|B) = P(B|A)P(A)/P(B)

Patient preferences the individual's evaluation of dimensions of health outcomes; factored into utility (value). Direct elicitation vs indirect assessment (prescored classification instruments, eg EQ-5D, SF-6D, 15D, Quality of Well-being Scale, HUI)
Utility standard gamble (choose between X time in illness vs therapy with a known risk of cure/death), TTO (time in state of illness vs time in state of perfect health), visual analogue (rate health states on a scale 0-100)
Time Trade-off TTO years of perfect health willing to live / 10 years of current health state. TTO * # of years in current health = QALYs
Standard Gamble SG utility of x years when having x years for sure equivalent to a gamble for 10 years vs 0 years with probability p = U
Quality Adjusted Life Year QALY 1 QALY is a year of perfect health; thus QALYs are expressed in terms of years lived in perfect health; weight values between 0 and 1 determined by TTO or SG

Cost effectiveness analysis Health benefit (e.g. QALY, lives saves, cases prevented) vs cost in dollars. Set a cost effectiveness threshold, eg. $50,000/QALY saved
Cost benefit analysis Compares both cost and benefit in dollars.
Incremental Cost/Effectiveness Ratio ICER one way to determine if a therapy is cost effective or not. Change in cost with change in a unit of effectiveness.

Receiver Operating Characteristic Curve ROC plot sensitivity (y) vs 1-specificity (x), i.e. true positives vs false positives. AUC higher with better test.
Relative Risk p(event|exposed) / p(event|no exposure)
False positive rate FPR 1-Sp
False negative rate FNR 1-Sn
Likelihood Ratio reflects both sensitivity and specificity; LR+ = Sn/1-Sp, i.e. TPR/FPR; LR- = 1-Sn/Sp, i.e. FNR/TNR
Accuracy (TP + TN)/ Total population

Applications of Clinical Decision Support CDS the process that provides clinicians, staff, patients or other individuals with knowledge and person-specific information, intelligently filtered or presented at appropriate times, to enhance health and health care.
Components of CDS Knoweldge base, Patient-specific information, Mode of communication: "active knowledge systems which uses two or more items of patient data to generate case-specific advice"

Functions of CDS Alerting, Reminding, Critiquing, Intepreting, Predicting, Diagnosing, Assisting, Suggesting
Rights of CDS (Osheroff) Right information, right person, right format, right channel, right time
Modes of delivery of CDS templated data-collection, suggestion, summarization, reminder, information, correct errors, recommend change in plan
Interruptiveness On demand (e.g link to fomulary); in-line or background (modeless, e.g. 'unread lab result'), popup, interruptive (modal, e.g. alerts, reminders requiring aknolwedgement)
Categories of CDS (leapfrog) 1. may use information about the current clinical context to retrieve pertinent online documents (infobutton) 2. may provide patient-specific, situation-specific alerts, reminders, physician order sets or other recommendations for direct action (classic CDS) 3. may organize information in ways that facilitate decision making and action

Users of decision support not just doctors, can be patients/families. Level of control: Pre-emptive, suppressible, hard-stop, interruptive

Implementing, evalution, maintaining decision support tools 10 Commandments for Effective CDS (Bates, 2003)
Evaluation of CDS Limitation of current literature, evidence of effectiveness limited. More evidence for process of care than patient outcomes. Diagnostic test ordering did well.


Knowledge generation Agreeing upon what constitutes relevant medical knowledge to be represented

Knowledge aquisition The process of importing knowledge into the CDS system (not the exact logic but the general flowchart of decision points and actions)
Knowledge acquisition tools facilitate entry of knowledge in a structured format directly by domain experts (eg Prot_g_, GEODE)

Knowedge modeling Representing on a granular level each piece of logic and resulting alert (from high-level format)
Ambiguity term can reasonably be interpreted in more than one way. Syntactic ambiguity (eg A or B and C). Semantic ambiguity (classic form, different interpretations of same word). Pragmatic ambiguity: usage (eg See you next Wednesday)
Vagueness boundaries of word's meaning are not well defined; word or phrase reduces level of information contained in a statement (eg 'may be appropriate'). Underspecification is vagueness that occurs when terms are used with insufficient details for definitive interpretation. Temporal vagueness (range from never to always). Probabilistic terms (impossible ... certain with vague terms in between (unlikely, probable). Passive voice (eg 'should be performed'). These uncertainties are established causes of decreased adherence to CPGs and practice variations despite apparent adherence. Strategies for vagueness include creating a controlled vocabulary or limited structured vocabulary, ordinal scale based on ranking, 'black-list' (eg possibly probably, should)
Tri-axial model for ambiguity and vagueness (Codish) 1. Semantics: pragmatic ambiguity when two or more recs are inconsistent or conflict; underspecification (eg moderate, adequate); temporal vagueness, probabilistic terms. 2. Source and rationale: inadvertant ambiguity reflects insufficient editing; deliberate vagueness poses a problem; quality of evidence and strength of rationale help but fail to provide clinicians with range of acceptable interpretations. 3. Recommendation component affected: what should be done, when it should be done, why it should be done
Computer Intepretable Guidelines must lend themselves to computation; representation format must allow for clinical expressivity (e.g. tempeoral dependencies, complex rules, strength of evidence, imperative/optional actions); ideally: standard vocabularies/semantics; interoperable, portable
DIKW Pyramid Data, Information, Knowledge, Wisdom: a class of models for representing puported structural and functional relationships

Knowledge representation Clinical algorithms, Bayesian statistics, Production rules, Scoring and heuristics
Clinical Algorithm The codification of evidence-based information into clinical pathways, practice guidelines, and decision rules. Branching algorithm; Path through flow chart; information nodes (gather), decision nodes; benefits: knowledge is explicit, easy to encode; limitations: must follow pathway, cant account for prior results, inability to pursue new etiologies; forerunner of modern CPG
Probabilistic Systems Use a naive Bayesian model and a sequential approach to calculate posterior probability = f(prior probability, new information). Assumes conditional independence of findings and mutual exclusivity of conditions, eg Leeds abdominal pain system (1975).
Leeds Abdominal Pain System de Dombal; Used sensitivity, specificity, disease-prevalent data for various signs, symptoms and test results to calculate using Bayes' theorem, the probability of seven etiologies of acute abdominal pain
Belief networks Conditional dependencies are modeled explicity; more modern approach
Production Rules Rule-based system; Knowledge encloded as IF-THEN rules. Require a formal language for encoding the rules plus an interpreter (inference engine) that operates on the rules. Limitations: depth first could lead to focus in wrong area; difficult to maintain rule bases; systems slow and time-consuming
Inference engines JESS (java-based); CLIPS (NASA); Drools (open-source, developed by JBoss community)
Backward chaining system pursues goal and asks questions to reach goal; inference engine determines whether the premise (left) of a given rule is true by invoking other rules that can conclude the values of variables that are currently unknown and that are referenced in the premise of the given rule
MYCIN Shortliffe (1975) first rule-based expert system; never used clinically; diagnosed infectious diseases (meningitis/bacteremia), backward chaining approach (unusual now)
Forward chaining similar to clinical algorithms.
Scoring and Heuristics Knowledge is represented as profiles of findings that occur in diseases; there are measures of importance and frequency for each finding in each disease; most scalable approach for comprehensive DSS
Scoring and Heuristics (examples) INTERNIST-1/QMR, DXplain, Iliad. Modern day: Isabel
INTERNIST-1 aimed to develop an expert diagnostician; evolved into Quick Medical Reference (QMR) where goal changed to using knowledge base explicity (did not succeed); DxPlain used principles of INTERNIST-1/QMR but developed more disease coverage. Evoking strength = given patient with this finding, how strongly should i consider the diagnosis, i.e. PPV; frequency: how often patients with the disease have the finding, i.e. sensitivity
HELP Salt Lake City (1970-80s); generated automated alerts; beginning of event-driven generation of specialized warnings - stored in HELP sectors, evolved into Arden Syntax
Arden Syntax HL7 standard representation format for decision support rules. International standard for MLMs endorsed by HL7 and ANSI. Maintenance, Library (eg purpose), Knowledge (mechanics), Resources (eg language)
Medical Logic Module MLM An individual decision rule in Arden Syntax. A specialized form of an ECA rule
Event-Condition-Action Rule ECA evaluation of situation-specific conditional expression logic is triggered by an external event - if true then action performed. Eg An abnormal resut is a condition in an event monitor with the event being the ordering of the test
Curly Braces Problem in Arden Syntax, area within curly braces, eg {get systolic blood pressure}. Local reference data contained within curly braces (fetch data out of target system); must translate into local implementation: a limitation in sharing MLM/CDS across systems
Guideline Interchange Format GLIF focuses on more complex multi-part guidelines, including complex clinical pathways
GELLO HL7 standard abstract expression language for specifying database queries (from Arden Syntax or others). Based on Object Constraint Language. Requires v3 RIM which has been slow to adopt
Shareable Active Guideline Environment SAGE First effort to separate CIS and CDS and recombine using an API: a vMR approach (solves the vocabulary problem)
Virtual Medical Record vMR HL7 standard based on the v3.0 RIM specifies canonical kinds of data one might find in EHR so that MLMs written in terms of GELLO queries on the vMR can be translated programmatically into actual queries on patient data at a local institution
SEBASTIAN places a standardized interface infront of CDS modules (on internet)
ATHENA-CDS Ontology-driven approach that separates static knowledge from problem solving knowledge (integrated into VistA). Driven by EON; ATHENA-CDS created an ontology of CPGs (eligibility criteria, clinical algorithm, guideline drugs) using Prot_g_, like a hierachy of classes in Ruby; the classes are instantiated to define the particular clinical algorithm, the particular guideline drugs etc. Other examples include GLIF, GUIDE, PRODIGY, Proforma, Asbru, GLARE
Controlled Terminology a list of terms that are the canonical representations of the concepts; contain relationships between the terms
ontology like a controlled terminology: enumerates the important entities and relationships among those entities (taxonomic relationships) as well as partitive relationships.

Knowledge management and maintenance Reliance on EHR patient data; guideline authorship so you can review/update, review patterns of use, commercial vendors, service-oriented architechure (SOA) for plug and play CDS
Decision Support Service DSS HL7 standard specification for a service-oriented architecture (SOA) to drive CDS
OpenCDS Uses DSS standard, open software tools; robust authoring environment for rules, integration with standard terminologies, can be integrated with other CDS tools such as HL7 infobuttons standard
CDS Workgroup reconcile various competing approaches to CDS; vMR is one project, a data model for representing input and output to CDS systems); Terminology management model (CTS2); Formalism for data manipulation in CDS (GELLO); formal method for describing process and work flow; taxonomy of services or actions evoked by guidelines

Legal, ethical and regulatory issues as long as CDS viewed as open loop (clinician between patient and system), not very regulated.
CDS Meaningful Use Requirements Objective 2: must satisfy measure 1 (implement 5 CDS interventions related to 4 or more CQMs) and measure 2 (drug-drug and drug-allergy interaction checks (excluded if EP writes fewer than 100 med orders))

Quality and safety issues recognized need to view health IT as potentially dangerous

Supporting decisions for populations of patients including used in personal health records

Evidenced-based Patient Care set of tools and disciplined approach to informing clinical decision-making

Evidence sources Hierarchy of Evidence (6S, bottom to top): single studies (MEDLINE) - synopses of single studies - syntheses (systematic reviews) - synopses of syntheses - summaries (textbooks, CPGs) - systems/actionable knowledge (guidelines, decision rules, order sets; integrate the lower levels with individual patients)
Level of evidence pyramid meta-analysis/systematic review > RCT > cohort > case-control > case reports/series > animal studies
Foreground questions asks about specific questions. Essential components (PICO --> PICOTS (time/setting)). Categories of questions : Intervention, Diagnosis, Harm, Prognosis. Results valid? important? applicable to patient care? subquestions?
Background questions ask about general knowledge
metanalysis and summary statistics metanalysis combines results of multiple similar studies; Odds Ratio for discrete variables, (weighted) Mean Difference for continuous variables; significance level = effect size * size of study
Cochrane Collaboration Intl collaboration produces systematic reviews/metanalyses

Evidence grading ACP smart medicine uses A, B, C (low-quality/anecdotal evidence) (there are many grading schemata)
United States Preventive Services Task Froce USPSTF AHRQ program for preventive services, grades A, B, C (selective), D (against), I (insufficient) with certainty levels (high, moderate, low)

Clinical Practice Guidelines CPG Systematically developed statements to assist practictioner and patient decisions about health care for specific clinical circumstances. "Systems" are evidence used in a CDS
Evidenced-based Practice Center EPC AHRQ program, reviews all relevant literature to produce evidence reports including HIT topics
National Guideline Clearinghouse NGC AHRQ initiative, bibliographic database provides information on CPGs to further their disemination and implementation; a carefully curated compilation of evidenced-based guidelines
Centre for Evidenced-Based Medicine CEBM U. Oxford EBM resource
National Institute for Health and Clinical Excellence NICE British system, approximately 100 CPGs are posted, summaries available
Digital Guidelines Library DeGeL facilitates gradual conversion of free text guidelines to a formal machine readable language
Guideline Element Model GEM ANSI standard; XML-based shared ontology for CPGs (for authors to annotate and identify key elements)


Information retrieval and analysis IR Basic concepts, content, indexing, retrieval, evaluation
Information retrieval basics Content indexed; indexing expressed as metadata. Users enter queries using metadata and search engine
Indexing Assigning of metadata to content items. Can assign subjects (terms), attributes (author, source, publication type...)
Human Indexing performed by professional indexer; follows protocol to scan resource and select terms from controlled vocabulary (most are hierarchical and have specific definitions for when term is to be assigned)
Automated Index index of ALL words that occur in content items; stop word list to remove common words; reduce plurals to stems
Medical Subject Headings MeSH most widely used controlled vocabulary; NLM to index articles from MEDLINE and PubMed. Hierarchical; 16 trees; 83 subheadings; over 26,000 terms and synonyms.
MEDLINE one of the first bibliographic databases to be available (1970s). References to biomedical journal literature. The original medical IR application (1971). Since 1997 MEDLINE is free via PubMed. Produced by the NLM. Over 22M references, over 5000 journals. About 3/4 M new references/year. Now there are links to full text. Indexed by professionals who follow a protocol. Assign 2-4 MeSH headings as central concepts and another 5-10 minor headings. Other metadata (author, source...)
PubMed NLM system for searching MEDLINE and related databases; includes some old MEDLINE. Based on Boolean heritage but has incorporated natural language searching; attempts to map to MeSH terms or author name; Advanced interface pretty unnecessary; default output order is reverse chronological; other features: spelling correction, graphical interface for limits, Link out to full text, clinical queries (search for EBM question types); with account (MyNCBI) can have saved searches, custom filters, emailed results. Pubmed Central (PMC) is free archive
Retrieval Boolean operators (set-based AND OR NOT), natural language queries (search engine maps words that are common to user's query and available content); + before word ignores that word; NEAR within 16 words
Term frequency-inverse document frequency tf-idf reflects how important a word is to a source, ie. tf*idf. tf is raw frequency of term in document; idf measures how much information the word provides: log(#documents / #documents that contain word)
Recall = Sensitivity # retrieved and relevant documents / #relevant documents
Precision = PPV # retrieved and relevant documents / #retrieved documents
Fall-out = false positive rate #non-relevant documents retrieves / #non-relevant documents available
F-measure a measure of accuracy : a weighted harmonic mean of recall and precision ( 2 precision*recall / (precision + recall)) from 0 to 1
Precision-Recall Curve PR Curve plot precision vs recall. Greater AUC is better
Knowledge-based Scientific Information Primary (original research in journals etc, re-analysis like meta analysis, sytematic reviews). Secondary (reviews, condensations, practice guidelines, handbooks)
Knowledge-based content Bibliographic, Full-Text, Annotated, Aggregated
Bibliographic content Original content of retrieval systems (old school, 1960s-70s), contains information about content. Bibliographic databases (MEDLINE, National Guidelines Clearinghouse). Web catalogs (websites that provide links to other websites). RSS. Contain metadata bout journal articles and other resources. US Government : MEDLINE (via NLM) and subsets like genomics information. Commercial publishers - EMBASE (part of SciVal), CINAHL
Really Simple Syndication / RDF Site Summary RSS Feeds provide short summaries about new content. Users receive RSS feeds by an RSS aggregator that can be configured for the site desired. Two versions : both provide title, link, description.
Resource Description Framework RDF General framework for how to describe any Internet resource such as website and its content, i.e. metadata.
Web Ontology Language OWL formal semantics; builds on RDF but adds semantics; expressed in triples
Web catalogs Aim to provide quality-filtered websites aimed at specific audiences. Distinction between catalogs and sites blurry. Some are aimed towards clinicains : HON Select, Translating Research into Practice (TRIP), others are aimed towards patients (Healthfinder)
Full-text content complete text, tables, figures, same as print version. Many published by Highwire Press, growing number available via open-access model (Biomed Central), Public Library of Science (PLoS), research grants pay for publication. Some publishers license and provide to vendors (Ovid, MDConsult). Books of a given publisher bundled into large collections (Access Medicine, Elsevier, LW&W); NLM has developed collection of books. Compendia of drugs, diseases, evidence and handbooks.
Annotated content Non-text or structured text annotated with text that describes what's in the content: image collections (prominent in visual specialties; Visible Human, Lieberman's eRadiology, WebPath, PEIR, DermIS, VisualDx), citation databases (databases of citations within articles. Science Citation Index + Social Science Citation Index = Web of Science <- Web of Knowledge; SCOPUS; Google Scholar), EBM databases (Cochrane Database of Systematic Reviews, evidence formularies: Clinical Evidence, JAMAevidence, PIER, UptoDate, Essential Evidence Plus which includes InfoPOEMS), CDS (order sets, rules, templates; growing market: Zynx, Provation), genomics databases(National Center for Biotechnology Information, part of NLM, produces; OMIM, Sequence databases - GenBank, Structure databases - Molecular Modeling Database, Genomes, Maps), other databases (ClinicalTrials.gov - not just for NIH trials, now a registry; NIH RePORTER - all research grants, replaced CRISP)
Aggregated content Aggregations for clinicians, eg Merck Medicus(Univadis); for researchers: Model organism databases eg Mouse Genome Informatics database. Consumer: MEDLINEplus - integrates a variety of licensed resources and public websites
Evaluation of retrieval systems Is system used? Users satisfied? Relevant information found (precision & recall)? Most studied is physicians : do they retrieve relevant information! May not capture full spectrum of usage

Search Skills Google performs an AND search of all terms


Workflow Sequence of physical and mental tasks performed by various people within and between work environments; can occur at several levels: one person; between/among people; across departments, facilities, and/or organizations. Can occur sequentially or simultaneously.
Workflow example: medication ordering Communication between provider and patient; clinical provider's thought process; physical action by the provider of writing a paper prescription and having the patient take the rx to the pharmacy, or the physical action by the provider of entering an electronic rx into an EHR and trasmitting the order electronically
Workflow analysis evaluation or assessment of a workflow process. An evaluation or assessment of the mental and physical steps of a workflow process, inclduing the order of the steps and the steps that comprise interaction among organizations involved. Evolved from the Scientific Management Movement (Taylor) in the late 1800s
Workflow analysis example: medication ordering 1. identify all types of communication in current use 2. analyze the steps and sequence that occur in each type of communication 3. thought processes : identify steps and sequence of steps that provider goes through; ask provider to share information with you 4. identify steps and sequence of steps to execute these processes(prescription), start to finish
Recurring Theme 1. Current state 2. Desired future state 3. Make a plan, execute 4. Evaluate outcome and consequences of being there
Principles of Workflow Analsysis 1. Goals of the work being studied must be understood 2. purpose and goals of the analysis must be clear 3. analysis of workflow must relate to the work as it is actually performed

Methods of workflow analysis Reduce a complex process into analyzable parts; describe complex clinical activity by identifying its component parts in a stepwise fashion
High-level flowchart diagram that provides a brief overview of a process only highlighting major events in the process
Detailed flowchart map that marks every step in a process, which includes decision points, waiting periods, and feedback loops
Swimlane flowchart map that displays processes carried out for multiple roles across multiple stages
Workflow Perpspectives People, Artifacts (eg paper record), Information, Tasks
Jordan's Data Collection Categories methods of data collection that may be used in a workflow analysis effort. Person-oriented record, Object-oriented record, Setting-oriented record, Task-oriented record
Person-oriented record recording done of workflow, eg to understand work of staff person in a specific role
Object-oriented record (artifact-oriented);eg trace path of a medical record document and understand its role in collaboration among staff
Setting-oriented record (event-oriented); eg emergency room or operating room workflow
Task-oriented record eg medication administration procedures
Workflow analysis study design identify what is being studied (person object setting task); identify the question you are trying to answer; capture only what you are looking for / establish boundaries; characterize which data you should collect ot best answer the study question
Methods of data collection qualitative (most common), quantitative, mixed, other
Observational data most common type of data in workflow analysis
Observational data (quantitative) eg data collected via observational systems (automated); collected via detached human observer eg counting
Observational data (qualitative) captures details of everyday work practices; ethnographic observation including participant observation : attends to meaning, goals, context, how people communicate
ethnographic observation human goes to clinical setting and observes workflow of clinical providers in that setting
Interviews and Focus Groups surveys, usually qualitative but can be quantitative
Observational fieldwork Describes the picture of having a human being conducting workflow analysis assessment enter into a clinical care delivery setting and beginning to collect information
Observational fieldwork - practical considerations 1. know and follow the rules of the setting in which you will be working 2. manage expectations of people you will be studying (explain pupose of analysis) 3. stay out of the way of people doing their work 4. give out project handouts with helpful information 5. plan ahead for your own supply and equipment needs
Analysis of workflow data type of analysis should be appropriate to the study question (eg studying safety or efficiency)
Task category identification record tasks and assign them into categories (eg collecting information from patient? Prior visits?)
Examine themes that emerge from data eg workaround that every person find themselves having to do
Grounded Theory an approach to analyzing workflow data and workflow analysis - group data according to task categories or themes and then apply a more structured approach to understanding the data, eg quantatiative approach to describing qualitative data

Principles of workflow reengineering Planned and deliberate 1. changes to mental and physical steps of people who move through a workflow process 2. changes to the steps in the interaction(s) among organizations involved in a workflow process TO CREATE workflow that supports improved (good, better, best) outcome of workflow activities (patient care delivery)
Workflow reeengineering The performance of a system is related overwhelming to the design of the system rather than to the intentions of the people who work in the system (Joseph Juran). System workflow, including the workflow of healthcare systems can be reengineered to improve design in order to improve quality and safety
High reliability in healthcare each patient receives the best quality care, every single time (AHRQ)
High-reliability organization HRO Have systems in place that are exceptionally consistent in accomplishing goals and avoiding potentially catstrophic errors. Eg airline industry, nuclear power industry among first to embrace HRO concepts.
Components of a workflow reengineering effort 1 1. Define performance goals : assess where organization needs to be (think regulatory standards) and wants to be (think excellence and customer satisfaction), in terms of performance of well-defined indicators. Relate goals to overall organizational performance goals
Components of a workflow reengineering effort 2 2. Assess / measure current organizational performance using all available reliable data
Components of a workflow reengineering effort 3 3. Prioritize goals (aka aims, objectives) & define timelines. Goals should be SMART (Specific, Measurable, Attainable, Realistic, Timebound)
Components of a workflow reengineering effort 4 4. Choose a methodology : CQI, TQM, performance improvement, Six Sigma, Lean, ISO, Baldridge, etc
Components of a workflow reengineering effort 5 5. Assemble a team to carry out the reengineering effort; document project team, goals, and responsibilities in a charter document
Components of a workflow reengineering effort 6 6. Map the current process : using, for example, workflow analysis methods
Components of a workflow reengineering effort 7 7. Revise the map (redefine the process to match the performance goal)
Components of a workflow reengineering effort 8 8. Implement the new process
Components of a workflow reengineering effort 9 9. Measure Outcome (Timeliness, Reliability, Efficiency, Staff satisfaction, Patient safety measures, clinical outcome measures)
Project RED training program designed to help hospitals re-engineer their discharge process
Culture of Improvement Workflow re-engineering can be performed in smaller systems within a larger healthcare organization; Organization as a whole needs to adopt a 'culture of improvement' to have a lasting impact on system performance

Pareto chart Frequency-sorted graph of events with cumulative percent line. Origin of the 80:20 rule. Used commonly to identify the most valuable targets for improvement
Five Whys Helps to identify root cause.
Total Quality Management TQM all-encompassing management approach to long-term success through customer satisfaction
Medical Errors (IOM) Diagnostic (delay in dx, failure to employ indicated tests, use of outmoded tests or therapy, failure to act on results), Treatment (error in performance of procedure or test, error in administering treatment, error in dose or method of using drug, avoidable delay in treatment, inappropriate care, Preventative (failure to provide prophylactic treatment, inadequate monitoring or follow-up of treatment), Other (failure of communication, equipment failure, other system failure)
Value Level of Quality / Cost
Error-Proofing Concepts Reason's swiss cheese model. Latent and active failures are the holes; processes, safeguards, and workflows are 'layers of cheese'; accidents/errors occur when the latent and active failures in different layers line up, allowing hazards to lead to losses
Failure Mode and Effects Analysis FMEA modes of failure can be risk-prioritized on three axes : 1. Severity of failure 2. frequency of Occurrence 3. Detectability
FMEA create flow diagram of process; for each step describe what happens if process fails and rank them in each of the three categories (1-5) (S none to fatal, O rare to common, D easy to difficult)
Risk Priority Number RPN S * O * D = ordinal measure (1 - 625)
Leading indicator (process measures) an indicator that anticipates future events, changes detectably before events occur (eg immunizations, antibiotics b4 surgery)
Lagging indicator (outcome measures) an indicator that follows an event (eg infections, VAP, complication rates)
Statistical Process Control SPC (Shewhart) Control Charts characterize fluctuation : 'common cause' - random, has no unnatural patterns. 'special cause' - falls outside UCL/LCL OR 'special cause' pattern
SPC charts (control or run charts) not a hypothesis test; when monitored over time, an indicator will fluctuate around an average value, defined by upper and lower control limits (3 standard errors ie 99%) Use 2 SE for upper and lower warning limits
Special cause patterns any single point outside 3SE, any 2/3 pts between 2-3SE, 4/5 consecutive pts beyond 1SE on same side, 8 consecutive on same side, 6 continually increasing/decreasing
Special cause terminology Shift (run of 6 or more pts on same side of center line), Trend (5 consecutive pts going in same direction), Run (too few or too many events crossing the center line), Cycle (periodicity in data suggests special cause), Pattern (cycles in data atributable to other factors besides time)
Cause-effect / Ishikawa / Fishbone diagrams Used with five whys to trace events back to root cause; represent outcome (head) and domains (bones)
Plan-Do-Study-Act PDSA small, repeated cycles to select targets, improve on a small scale, implement widely, and measure outcome. Steps: form team, set aims (time specific and measurable), establish measures (should be good indicators), select target for change (use FMEA, Pareto, Fishbone, other techniques to identify targets)
Healthcare Effectiveness Data and Information Set HEDIS non-governmental Quality Monitoring Program supported by America's health plans (maitained by NCQA)
Six Sigma 3.4 defects per million (99.999% error free) : DMAIC (define, measure, analyze, improve, control) or DMADV for new processes (define, measure, analyze, design, verify). Focus on quality; seeks to remove effectively any defect or error from a process
Toyota Lean and related strategies Remove all non-value added activities: Muda (uselessness, wastefulness), Mura (irregularity, unevenness), Muri (unreasonable, burdensome work)
Muda overproduction/underproduction, inventory problems, repairs/rejects, motion waste, processing waste, waiting, transport
Value stream mapping (within Lean) : graphical depiction of inputs, throughputs, outputs of process to highlight opportunities for improvement. Frontline staff bring forth ideas
Kaizen / Continuous Quality Improvement CQI "change for the better" : small improvement, rapid adaptation to results. Standardize operational activities; measure operation; compare measurements to requirements; engage frontline staff in identifying opportunities to improve; when improvements work, make them the new standard; repeat
Lean supporting conventions Kanban cards (visual indicators that supply is empty); Andon (visual indication that indicates production status / alerts when assistance is needed; Poka-yoke (mistake avoiding in design or process; eg intentional incompatibility of refill spouts for inhaled anesthetics, color-coding of medical gases, SIM card)