Print bookPrint book

Course Content

Site: SLMTA
Course: Statistical Quality Control and Method Validation/Verification (QCMV2) e-Learning
Book: Course Content
Printed by: Guest user
Date: Saturday, 21 December 2024, 2:57 PM

Table of contents

0 - Workshop Introduction

Overview

Purpose

Participants are provided with an overview of how this workshop integrates quality control (QC) into their laboratory to support their accreditation efforts.  Additionally, workshop expectations are presented to participants.

Key Message

  • The workshop targets primarily quantitative QC and method evaluation.
  • Linkages between QC workshop training and ISO 15189 will be made throughout the workshop.
  • Where are we, where do we expect to be, and where do we want to be are three questions that will be used throughout the workshop to assist with data analysis and evaluation.

Self Assessment

Can you: 

  • Understand how the QC workshop supports the SLMTA process?
  • Understand the workshop’s scope, targeting quantitative QC?
  • Understand expectations during and after the workshop?

Resources



1 - Let's Examine the Basics

Overview

Purpose

In this activity, the pre-requisite reading’s Final Examination on basic QC and Measures of Central Tendency worksheet will be reviewed with participants.  Tools to calculate mean, standard deviation (SD), and coefficient of variation (CV) will be introduced and used throughout the workshop.

Key Message

  • The mean, SD, and %CV are foundational statistics used with QC
  • Given a data set, the mean, SD, and %CV can be calculated.
  • Calculators, spreadsheets, and online resources are available tools sites can use with their QC data.

Self Assessment

Can you: 

  • Calculate the mean, SD and %CV from a given data set?
  • Utilize the tools introduced to obtain a mean, SD, and %CV from their site’s data set?
  • Calculate the measures of central tendency?

Resources


2 - Gaussian is the key

Overview

Purpose

Stable analytical systems will produce the same Gaussian distribution of data.  When a system undergoes a change, an unexpected data point will be produced.  In this activity, participants will demonstrate the foundational principles of statistical QC (SQC).

Key Message

  • The accuracy and precision of an analytical system describes WHERE WE ARE.
  • The basis of SQC is knowing WHERE WE ARE and WHERE WE EXPECT TO BE.
  • If WHERE WE ARE with our current observed value is not WHERE WE EXPECT TO BE, then a change has occurred.

Self Assessment

Can you: 

  • List the performance specifications that describe WHERE WE ARE with an analytical system?
  • Define SQC and how it applies to monitoring their analytical system?
  • Define the criteria required for SQC to monitor change?

Resources



2.1 Gaussian is key


2.2 Measures of Central Tendency


2.3 Gaussian distribution

2.4 Gaussian demonstration

2.5 Levey Jenings Chart


2.6 Data populations


2.7 SQC recap


3 - Match it up

Overview

Purpose

QC rules alert laboratorians to a change in the analytical system (measurement procedure). In this activity, participants match charts to the correct QC rule violation and identify the type of error.

Key Message

  • Results of QC samples are analyzed to assess and alert us to changes in accuracy and precision (method performance).
  • The nomenclature for rule violations includes the number of data points involved and the applied control limits usually expressed as the mean ± a multiple of the SD.
  • Different rules indicate if the change is random or systematic.  This information is used to assist with troubleshooting an out-of-control run.

Self Assessment

Can you: 

  • Analyze L-J charts and Gaussian distributions and identify the rule violation?
  • Determine if the rule indicates a systematic or random error?
  • Understand how SE and RE assists with troubleshooting?

Resources



3.1 Match it up Introduction



3.2 Common QC terms


3.3 Homework Part 1 Match it up


3.4.1 Homework Part II Introduction Match it up


3.4.2 Homework Part II Match it up


4 - The Front line worker

Overview

Purpose

Front-line workers assess whether or not an analytical run is acceptable.  QC protocols developed by laboratory management must standardize the decision criteria used by the workers.  In this activity, participants are provided with QC charts and a Westgard Multirule algorithm and must apply the decision-making process to determine acceptability of the analytical run.

Key Message

  • Multirules provide a high level of error detection capability while at the same time keeping false rejection rates low.
  • Laboratory management must provide clear instructions to the front-line workers regarding QC decision criteria.
  • The 1:2s warning rule is a scanning tool that can save time and effort in manual QC applications.

Self Assessment

Can you: 

  • Determine whether an analytical run is acceptable or not based upon a given QC protocol?
  • Recognize how the 1:2s warning rule can be used as a scanning tool?
  • Understand when a multirule should be applied to a measurement procedure?

Resources



4.1 Introduction to The Front line worker


4.2 Homework The Front line worker

5 - It Begins with the Right Chart

Overview

Purpose

Using the QC tool, L-J Chart, one can visually inspect a quantitative method’s current accuracy and precision to quickly assess if a change is occurring within the system.  In this activity, participants learn how to correctly create this tool, how to visually assess it, and how to avoid pitfalls commonly encountered with its set-up.

Key Message

  • One of the most widely used tools with QC is the L-J chart.  This tool allows one to visually assess for a change in a method’s current performance.
  • When L-J charts are regularly examined, a small change may be recognized long before a QC rule violation occurs.
  • If the assigned values used on the chart do not reflect the observed values of the current performance, then the QC rules are unable to function properly to detect errors.  The data will no longer appear normally distributed.

Self Assessment

Can you: 

  • Create a properly prepared L-J chart?
  • Visually assess a chart to see if there has been a change in the method’s performance?
  • Recognize common pitfalls when the assigned values do not equal the observed values of a stable system?

Resources


5.1 Introdcution It Begins with the Right Chart


5.2 True and False Accepts and Rejects


5.3 LJ Chart set up


5.4 Homework Part I Assigned=Observed

5.5 Homework Introduction to Part II Assigned ≠Observed


5.6 Homework Part II Assigned ≠ Observed


6 - Paralell testing

Overview

Purpose

Only when the system is known to be stable can data be collected to calculate the observed mean and SD of a new lot number of QC material. % CV is introduced and used in the SD computation of short shelf-life control materials.

Key Message

  • New lot numbers of control material should be tested in parallel with the currently in-use control material.
  • Parallel testing is only performed when the instrument is operationally stable.
  • Package inserts serve as a guideline only and should not be used when creating the L-J chart for the new control.  %CV tells us WHERE WE ARE in terms of precision.

Self Assessment

Can you: 

  • Understand the importance of parallel testing?
  • Identify the steps to perform parallel testing at their site?
  • Calculate the %CV?

Resources



6.1 Paralell testing QC material



6.2 Homework CV%


7 - Total Error

Overview

Purpose

Total error (TE) combines bias and imprecision to quantify the largest variation from the true (target) value.  In this activity, participants are introduced to bias and true value.  Using the information supplied by 3 key numbers, participants calculate the TE in units and percent.  Additionally, participants explore the impact that bias and imprecision can have on TE and relate site-specific processes that affect TE to checklist items.

Key Message

  • 3 key numbers are required to calculate TE – mean, SD, and true value.
  • The target value is our best estimate using sources that each has its own limitation.
  • TE is the total variation of our value from the true value.

Self Assessment

Can you: 

  • Calculate bias and absolute bias?
  • Calculate TE in units and percent?
  • Identify ways to reduce bias and imprecision at their site?

Resources


7.1 Introduction to Total Error


7.2 Bias equation TE


7.3 TE Calculation Part I


7.4 TE Calculation Part II


7.5 Homework Calculating Total Error


8 - Putting TEa into Quality

Overview

Purpose

To control quality, one must first determine what quality is needed for a test.  In this activity, participants will explore what information the fourth key number must provide to define the quality required.  Additionally, participants will use a laboratory scenario to assess if the method remains acceptable for the intended clinical use of the test result.

Key Message

  • TEA defines the quality specifications specific to the analyte being tested to make the analytical result clinically meaningful.
  • QC statistics can effectively reflect method performance if acceptable performance limits are defined and applied.
  • For clinically meaningful results, the TE should be less than the TEA for that analyte.

Self Assessment

Can you: 

  • Identify the four key numbers needed for quality?
  • Calculate error limits if given the Target Value and TEA?
  • Compare TE to TEA to determine if the method still performs within acceptable limits?


Resources



8.1 Homework Intended use Putting TEa into Quality


8.2 Using Total Allowable Error


8.3 Calcuating TEa limits


8.4 Homework TEa Calculations and Reagent lot number changes


9 - How Far can Your Mean Shift

Overview

Purpose

Benchmarking measurement procedures using TEA and the current method’s performance allows a laboratory to identify which methods are meeting quality specifications and which are not. In this activity, Sigma-metric and Critical Systematic Error (ΔSEc) are introduced.  Participants will review a site’s monthly summary report and perform an investigational analysis using instrument records and QC charts.

Key Message

  • SEc indicates how far the mean can shift before quality performance requirements are exceeded.
  • Benchmarking methods using a Sigma scale allows a laboratory to use their resources wisely.
  • Good recordkeeping is essential in resolving QC problems.

Self Assessment

Can you: 

  • Describe how SEc and Sigma indicate method performance relative to quality performance goals?
  • Calculate SEc and Sigma?
  • Use information gleaned from records to troubleshoot a QC issue?

Resource



9.1 Introduction to How Far can Your Mean Shift


9.2 Introduction to SEc


9.3 Calculating SEc

9.4 Introduction to Sigma metrics


9.5 Homework Calculating SEc and Sigma


9.6 Homework Review and investigate


10 - How to Select Control Rules

Overview

Purpose

Laboratories must select appropriate QC rules based on the quality required for a test and the method’s observed accuracy and precision.  In this activity, participants are introduced to the Sigma-metrics QC Selection Tool, available at Westgard.com.  Using a case scenario, participants will select appropriate QC rules and design a QC strategy for a workstation.

Key Message

  • The measurement procedure’s quality requirement and the accuracy and precision must be considered in the selection of appropriate QC rules.
  • The control rule selected should alert the laboratorian to a significant change before wrong results are reported.
  • Selected control rules should maximize the error detection (Ped ≥ 90%) and minimize the false rejection (Pfr ≤ 5%) while using the lowest number of control measurements possible.

Self Assessment

Can you: 

  • Select appropriate QC rules using the Sigma-metrics QC Selection Tool from the Westgard internet site?
  • Select a TEA% using the Westgard internet site?
  • Design an effective QC strategy for a workstation?

Resources




10.1 Introduction to How to Select Control Rules

10.2 QC selection tool introduction Part I

10.3 QC selection tool introduction Part II

10.4 Homework Predicting Probabilities

10.5 Homework Workstation activity


11 - How Proficient Are We

Overview

Purpose

External Quality Assessment Schemes (EQAS) enable the laboratory to assess on-going accuracy by comparing its method’s performance to external sources.  In this activity, participants are introduced to standard deviation index (SDI) and z-score and will review a proficiency test (PT) report.

Key Message

  • Proficiency testing (PT) programs provide a way to verify long-term accuracy of a method.
  • SDI is a quick indicator of how well our mean compares to the method peer mean.
  • All PT failures must be investigated and documented by the laboratory.

Self Assessment

Can you: 

  • Recognize the common elements found in a PT report and their significance?
  • Calculate SDI and use that information to assess the accuracy of the method?
  • Recognize that PT results, both acceptable and unacceptable, should be evaluated?

11.1 Why EQA Part 1


11.2 Why EQA Part 2


11.3 PT Programs Part 1


11.4 PT Programs Part 2


11.5 PT Programs Part 3


11.6 PT Programs Part 4


11.7 PT Programs Part 5


11.8 Introduce the Activity


12 - Understanding Interlab Comparison

Overview

Purpose

Laboratories using the same measurement procedure and analyzing the same control material will generate similar means and SDs.  This information is a valuable source to determine the target value of the QC material for a laboratory and compare their method’s performance to their peers.

In this activity, participants are introduced to coefficient of variation index (CVI).  Based on the information supplied from the inter-laboratory comparison report, participants will investigate a QC problem.

Key Message

  • Inter-laboratory peer comparisons can help identify method and PT problems.
  • The peer mean is an excellent source of the true (target) value for each control.
  • Coefficient of Variation Index is a peer-based metric of imprecision.

Self Assessment

Can you: 

  • Calculate the SDI and CVI?
  • Verify the 4 key numbers used to monitor and evaluate performance?
  • Perform an investigational analysis regarding a summary statistic flag?

Resources



12.1 Introduce


12.2 Comparison


12.3 QC Materials


12.4 How It Works


12.5 CVI Part 1


12.6 CVI Part 2


12.7 CVI Part 3



12.8 Introduce the Activity


13 - Introduction to Method Validation and Verification

Overview

Purpose

Before a method can be placed into routine service, it must be evaluated to ensure that the measurement procedure meets defined criteria, such as sensitivity, specificity, precision, accuracy, and linearity.   In this activity, participants will perform method verification experiments and assess data to determine if the method is acceptable or not.

Key Message

  • Even though manufacturers test their methods extensively, there are many factors in an individual laboratory that can affect the actual precision and accuracy of the method.
  • Method evaluation is performed to reveal the amount of error present in the new method.
  • Once the performance of the method has been judged to be acceptable from method evaluation studies, then statistical QC procedures need to be selected that can detect medically important errors.

Self Assessment

Can you: 

  • List the steps required to introduce a new method?
  • Collect and analyze the experimental data needed to quantify the error present?
  • Judge if the method meets pre-defined quality requirements?
  • Verify a reference interval?


Resources



13.1 Introduction Part 1


13.2 Introduction Part 2

13.3 ISO Requirements


13.4 Intro Summary


13.5 Performance Characteristics 1

13.6 Performance Characteristics 2

13.7 Performance Characteristics 3

13.8 Performance Characteristics 4

13.9 Performance Characteristics 5

13.10 Validation and Verification Summary

13.11 Introduce the Activity - Decision Tree

13.12 MV Overview Part 1

13.13 MV Overview Part 2

13.14 MV Overview Part 3

13.15 Precision Experiment 1

13.16 Precision Experiment 2

13.17 Intro the Activity - Precision

13.18 Linearity Experiment 1

13.19 Linearity Experiment 2

13.20 Linearity Experiment 3

13.21 Linearity Experiment 4

13.22 Linearity Experiment 5

13.23 Linearity Experiment 6

13.24 Linearity Experiment 7

13.25 Intro the Activity-Linearity

13.26 Accuracy Experiment 1

13.27 Accuracy Experiment 2

13.28 Accuracy Experiment 3

13.29 Accuracy Experiment 4

13.30 Accuracy Experiment 5

13.31 Accuracy Experiment 6

13.32 Accuracy Experiment 7

13.33 Accuracy Experiment 8

13.34 Accuracy Experiment 9

13.35 Accuracy Experiment 10

13.36 Accuracy Experiment 11

13.37 Introduce the Activity - Accuracy

13.38 Reference Interval Experiment 1

13.39 Reference Interval Experiment 2

13.40 Reference Interval Experiment 3

13.41 Reference Interval Experiment 4

13.42 Introduce the Activity - Ref Interval

14 - Measurement of Uncertainty

Overview

Purpose

Every result has some degree of uncertainty that comes from the calibration of the method and the analysis itself. In this activity, participants learn how to determine measurement uncertainty using a top-down approach and how to apply measurement uncertainty in an advisory capacity for their clients.

Key Message

  • The uncertainty gives the limits of the range in which the true value of the measurand is estimated to be at a given probability.
  • The long-term %CV or SD data gained through the laboratory’s IQC is sufficient for satisfying the minimal criteria for MU determination.
  • The MU is helpful in differentiating whether a change in a serial test result is due to analytical variation or a true physiological change.

Self Assessment

Can you: 

  • Recognize the need for hospital management support and analytical staff adherence when planning a QC program?
  • Identify the steps needed during the planning phase to successfully implement a QC program?
  • Further refine the process plan to identify what happens, who’s responsible, and by when?


Resources



14.1 Introduction


14.2 ISO Requirements

14.3 TE vs MU

14.4 MU Part 1

14.5 MU Part 2

14.6 MU Part 3

14.7 Introduce the Activity - MU Calculation

14.8 Traceability 1

14.9 Traceability 2

14.10 Commutable



14.11 Apply MU


14.12 Diagnostic Uncertainty



14.13 Intro the Activity - Reliable Repeat



14.14 Intro the Activity - Using the Rule of 3


15 - Improvement Project Master Class

15 - Improvement Project Master Class

15.1 Improvement Project Master Class

  • Overview

Purpose

The knowledge gained from a workshop becomes effective when the improvement is applied at the site for better patient care. In this activity, participants will map the process needed to perform an improvement project (IP) assignment.

Key Message

  • The planning phase is essential in any improvement project.
  • The support of hospital management and analytical staff is essential for implementing an effective QC program.
  • The QC IP plan involves many sections of the accreditation checklist.

Self Assessment

Can you: 

  • Recognize the need for hospital management support and analytical staff adherence when planning a QC program?
  • Identify the steps needed during the planning phase to successfully implement a QC program?
  • Further refine the process plan to identify what happens, who’s responsible, and by when?