Module 5 · Lesson 14 of 17
Evaluating pilot results
This lesson and the entire Lean Six Sigma Black Belt academy are free of charge — no fees, ever.
Learning objectives
- Explain the purpose of evaluating pilot results in a garment-factory improvement project.
- Apply the described method to a representative shop-floor situation.
- Recognize the common mistakes and how to avoid them.
Concept
Pilot evaluation combines statistical evidence, practical effect, process stability, risk, and implementation learning.
Plot time order first, check measurement comparability, then calculate effect and confidence interval.
Select paired, independent, or proportion test as appropriate; guardrail metrics matter as much as the primary Y.
Variables, units, and assumptions
Primary Y (defects/unit or minutes), guardrails (output/hr, ergonomic score), period (dates).
Comparable measurement, similar style mix pre/post, no confounding change.
Free-spreadsheet workflow
Free spreadsheet: paste pre/post data, plot run chart, compute mean/SD, 95% CI on difference (t-based), stratify by shift.
Interpretation and limitations
Effect exceeds CI half-width AND meets practical threshold AND passes guardrails → confirm.
Short pilots may miss seasonality; guardrails may reveal delayed effects.
Garment-factory example
Compare FPY and output per hour on pilot and comparable lines while stratifying by style and shift.
Method
- Verify protocol fidelity.
- Clean data.
- Graph in time order.
- Estimate effect and CI.
- Test assumptions.
- Calculate financial impact.
- Summarize operational lessons.
Common mistakes
- Relying only on p-value while ignoring practical effect.
- Ignoring negative effects on output or ergonomics.
Knowledge check
Pick one answer per question. Explanations appear after you submit.
1. Statistical significance without meaningful practical improvement is:
2. The first analysis step is:
Author: NOSK.life · Last reviewed: 2026-07-21