A Painless Look at Using Statistical Techniques to Find the Root Cause of a Problem
Contributor: From the Desk of the Executive Editor
Posted: 06/01/2009 5:45:00 PM EDT | 6
Editor's note: There are three concepts that must be understood by all Six Sigma practitioners—namely, homogeneity, detecting significant variation in performance metrics and disaggregating non-homogeneous data into meaningful subgroups.
Sound complicated? Probably. But this article hopes to demystify these highfalutin statistical notions and illustrate how they interrelate when attempting to find the root cause of an identified problem.
Stay with it. It's really painless. And if you understand what is said, you're going to be well on your way to achieving real success in Lean Six Sigma continuous improvement programs. And for more information, read the article "Turning Customer Complaints into Profitable Opportunities Using the Six Sigma Tool Kit."
The business press is not aware of it. Neither is corporate America. National magazines deny its existence. And public service organizations think it’s a process by which to extend the shelf life of milk.
It is called homogeneity. It is seldom mentioned in most Six Sigma continuous improvement seminar, claiming to be “practical, down-to-earth and hard-hitting.” Yet without an understanding of this concept called "homogeneity," most continuous/quality improvement efforts are impossible.
Before defining the homogeneity concept, let’s define the purpose of statistical methods. Here are some of the best definitions:
Preface to Homogeneity
Everyone talks about performance measurements. We will discuss what to measure and how to measure in a future issue. But for now, it is important to understand the difference between a descriptive statistic that is a measure of performance and a performance measurement methodology.
Descriptive measures do not lead to improvement in productivity. Take, for example, statistics on accidents: They tell you about the number of accidents in the workplace, in the home and on the road.
Indeed, one can even determine if there is a trend in the frequency of accidents using data collected on accidents over time. But this statistic cannot, nor does it pretend to, tell you how to reduce the number of accidents.
A statistical-based performance measurement analysis methodology, if properly used, can locate the root cause of, say, accidents and point the way to taking corrective action in reducing their frequency.
This “descriptive statistic” has its uses. But you would be terribly irresponsible and totally misguided if you thought a descriptive statistic provides a prescriptive answer.
In short, what can be done to lower the accident rate? There is a world of difference between “knowing” a problem exists and “doing” something about it.
Is Your Data Overly Aggregated?
Assume it is! You can never go wrong with this mind set. Although most people are not aware of it, the key to finding the root cause of many problems is to “disaggregate” an aggregate number.
Find relevant sub-aggregates or sub-groups. This requires thinking and knowledge of a particular situation. The average SAT scores of a given university can be subdivided by different schools within the university—namely, the engineering school, the business school, the nursing school and the like.
Instinctively, you know there will be variation in SAT scores among the different divisions or schools within the university. The many specialty divisions/schools of the university are our subgroups in this situation.
Most people, when given this illustration, usually respond with: "I knew that." And we do not doubt them. Sounds so easy.
Yet in many organizational situations, the simplicity of finding relevant subgroups is elusive. People jump to the wrong conclusions because they fail to think through the relevant subgroups.
A Quick Digression
Before the advent of high-speed computing, the “aggregation” problem was an impediment to management analysis, planning and control. Computers eliminated the aggregation problem (or so it seemed).
Data could now be obtained at the correct level of aggregation. That it could be obtained didn't mean it was obtained. For 50 years, information technology has centered on data—their collection, storage, transmission and presentation.
The focus was on the "T " in IT. Today's new emphasis is on the "I." Lean Six Sigma forces organizations to ask, "What is the meaning and purpose of information?"
Back to Our Discussion...Subdivide, Subdivide, Subdivide
The percentage of delayed flights for a given airline is an “aggregate number.” If delayed flights are subdivided into groups representing delayed flights at five airports, the resulting set of numbers represent “sub-aggregates” or “partitions.”
The degree of aggregation in data refers to the level of detail or refinement in data. A high level of aggregation conceals differences between and among subgroup categories.
Most people realize “analysis of disaggregated data” may reveal important problems. For example, if the number of delayed flights increased in the aggregate, the number of delayed flights may have decreased in four out of the five airports. One airport may account for the overall increases in delayed flights because of, say, extremely bad weather conditions or other assignable causes.
This increase in delayed flights, if offset by a decrease in delayed flights in other airports, “conceals what’s happening” and prevents remedial action if information Is presented in a highly aggregated form. (Without going into detail in this article, this is the core idea underlying Pareto analysis—one of the most powerful tools in the Lean Six Sigma toolbox.)
Looking for Significant Differences Between and Among Subgroup Categories
Let's take a very simple example. Someone tells you, "The invoice error rate equals 10 percent." You shake your head and mumble something to the effect of, "That seems rather high."
The basic question is whether the aggregate number or percentage (the result for the total group) conceals differences among subgroups. For example, can the aggregate statistic,10 percent incorrect invoices—be subdivided into sub-groups?
Definitely. One basis for subdivision is the two shifts doing the invoicing. For example:
Table One
This subgrouping reveals a significant difference in invoice error rates. All incorrect Invoices were made during the second shift (20 percent). Since the number of invoices processed is equal, it is unnecessary to use a weighted average to arrive at the average Invoice processing error rate. In this case, we add the invoice error rates and divide by the total number of rates (0 percent + 20 percent /2 = 10 percent).
This Process Can Be Extended to Several Additional Subdivisions
By introducing additional subdivisions, and finding significant differences or variations among subgroups, we can move closer to determining the “root cause” of the incorrect invoices.
For example, the experience level of the employees working the two shifts supplies a possible explanation for the excessive number of the incorrect invoices. In this organization it was discovered that all new workers were routinely assigned to the second shift.
Table Two
Table Two indicates that new workers were responsible for all the incorrect invoices, if the subdivision process stopped at this point. It would appear that more intensive training of new billing staff could dramatically reduce the rate of incorrect invoices.
However, if we introduce one additional basis for subdivision, we learn more about the “root cause” of Incorrect invoices. What is the composition of the new invoice people on the second shift? That is, can the subgroup be subdivided on the basis of some other characteristic yet not under investigation?
Table Three indicates that inexperienced billing staff are of two kinds: 1) full-time employees and 2) part-time temporary staff.
Table Three
Table Three reveals that part-time temporaries are the cause of the incorrect invoices. Why? The full-time staff received thorough training in the invoicing policies and procedures.
But—and this is a very big "but"—the part-timers are brought in during peak shipping periods and are often thrown at the problem, receiving no training. That this happens shouldn't surprise you. However, it often gets overlooked when confronted with aggregated numbers.
People see what is presented to them; what is not presented tends to be disregarded. And what is presented, more often than not, are "problems couched in aggregate data"—especially in the areas where performance falls below expectations—which means that managers tend not to see the potential cause of the problem.
The Root Cause, Once identified, Can Be Eliminated via Management Action
How can the number of incorrect invoices be eliminated? Less dependence on part-timers in critical areas, and more effective training of the temporaries that are used. This particular organization stopped hiring temporaries for billing and invoice costs dropped eighty percent.
Management must change its policies. In this case, the hiring of office temporaries was eliminated. Management took an action to remedy the problem.
There was a corresponding savings in the customer support area. Customers no longer had to deal with chronically incorrect bills. (This example is a bit oversimplified but it is based on an actual case. Jim Harrington, a Deming missionary who worked at IBM for 35 years, estimates that 50 percent of the costs of every billing system are attributable to “screw ups." Why continue to pay that money?)
Now We Can Define Homogeneity
The critical question: Can the data be subdivided on the basis of some characteristic other than the one under investigation, so as to reveal differences among subgroups?
If it is Impossible to further sub-divide your data, the data can be classified as “homogeneous." If, however, one discovers a basis for subdividing which reveals differences, the data set is “heterogeneous” or “non-homogeneous.”
Aggregation has the effect of ignoring relevant subgroups. Conclusions that seem obvious when we look only at aggregated data become debatable when we examine "lurking variables" or relevant subgroups.
Homogeneity, Variability, and Statistical Procedures
Let’s review:
Statistical procedures detect significant variation among subgroups. If significant differences in a performance characteristic (because of thoughtful subdivision of a data set) are found to exist, the reasons for the variation must be investigated. After the “causes” of the variation are discovered and eliminated, the performance measurement improves.
Posted: 06/01/2009 5:45:00 PM EDT | 6
|
|
| Special note: We are responding to reader requests concerning the development of a solid foundation for understanding, in very practical terms, statistical analysis. Many people suffer from acute statistical trauma. We are introducing a continuing series of articles that, we hope, provides readers with the basics required for proper usage of statistical tools and methods. The first four "back-to-basics articles" should be read in the following order—namely: 1) Preface To Performance Measurement: Understanding Ratios, 2) Ratios: Abuses and Misuses, 3) Averaging Ratios and The Perils of Aggregation and 4) A Painless Look At Using Statistical Techniques To Find The Root Cause of a Problem. |
Sound complicated? Probably. But this article hopes to demystify these highfalutin statistical notions and illustrate how they interrelate when attempting to find the root cause of an identified problem.
Stay with it. It's really painless. And if you understand what is said, you're going to be well on your way to achieving real success in Lean Six Sigma continuous improvement programs. And for more information, read the article "Turning Customer Complaints into Profitable Opportunities Using the Six Sigma Tool Kit."
The business press is not aware of it. Neither is corporate America. National magazines deny its existence. And public service organizations think it’s a process by which to extend the shelf life of milk.
It is called homogeneity. It is seldom mentioned in most Six Sigma continuous improvement seminar, claiming to be “practical, down-to-earth and hard-hitting.” Yet without an understanding of this concept called "homogeneity," most continuous/quality improvement efforts are impossible.
Before defining the homogeneity concept, let’s define the purpose of statistical methods. Here are some of the best definitions:
- Making sense out of variation
- Dealing with variation
- The study of variation
Preface to Homogeneity
Everyone talks about performance measurements. We will discuss what to measure and how to measure in a future issue. But for now, it is important to understand the difference between a descriptive statistic that is a measure of performance and a performance measurement methodology.
Descriptive measures do not lead to improvement in productivity. Take, for example, statistics on accidents: They tell you about the number of accidents in the workplace, in the home and on the road.
Indeed, one can even determine if there is a trend in the frequency of accidents using data collected on accidents over time. But this statistic cannot, nor does it pretend to, tell you how to reduce the number of accidents.
A statistical-based performance measurement analysis methodology, if properly used, can locate the root cause of, say, accidents and point the way to taking corrective action in reducing their frequency.
This “descriptive statistic” has its uses. But you would be terribly irresponsible and totally misguided if you thought a descriptive statistic provides a prescriptive answer.
In short, what can be done to lower the accident rate? There is a world of difference between “knowing” a problem exists and “doing” something about it.
Is Your Data Overly Aggregated?
Assume it is! You can never go wrong with this mind set. Although most people are not aware of it, the key to finding the root cause of many problems is to “disaggregate” an aggregate number.
Find relevant sub-aggregates or sub-groups. This requires thinking and knowledge of a particular situation. The average SAT scores of a given university can be subdivided by different schools within the university—namely, the engineering school, the business school, the nursing school and the like.
Instinctively, you know there will be variation in SAT scores among the different divisions or schools within the university. The many specialty divisions/schools of the university are our subgroups in this situation.
Most people, when given this illustration, usually respond with: "I knew that." And we do not doubt them. Sounds so easy.
Yet in many organizational situations, the simplicity of finding relevant subgroups is elusive. People jump to the wrong conclusions because they fail to think through the relevant subgroups.
A Quick Digression
Before the advent of high-speed computing, the “aggregation” problem was an impediment to management analysis, planning and control. Computers eliminated the aggregation problem (or so it seemed).
Data could now be obtained at the correct level of aggregation. That it could be obtained didn't mean it was obtained. For 50 years, information technology has centered on data—their collection, storage, transmission and presentation.
The focus was on the "T " in IT. Today's new emphasis is on the "I." Lean Six Sigma forces organizations to ask, "What is the meaning and purpose of information?"
Back to Our Discussion...Subdivide, Subdivide, Subdivide
The percentage of delayed flights for a given airline is an “aggregate number.” If delayed flights are subdivided into groups representing delayed flights at five airports, the resulting set of numbers represent “sub-aggregates” or “partitions.”
The degree of aggregation in data refers to the level of detail or refinement in data. A high level of aggregation conceals differences between and among subgroup categories.
Most people realize “analysis of disaggregated data” may reveal important problems. For example, if the number of delayed flights increased in the aggregate, the number of delayed flights may have decreased in four out of the five airports. One airport may account for the overall increases in delayed flights because of, say, extremely bad weather conditions or other assignable causes.
This increase in delayed flights, if offset by a decrease in delayed flights in other airports, “conceals what’s happening” and prevents remedial action if information Is presented in a highly aggregated form. (Without going into detail in this article, this is the core idea underlying Pareto analysis—one of the most powerful tools in the Lean Six Sigma toolbox.)
Looking for Significant Differences Between and Among Subgroup Categories
Let's take a very simple example. Someone tells you, "The invoice error rate equals 10 percent." You shake your head and mumble something to the effect of, "That seems rather high."
The basic question is whether the aggregate number or percentage (the result for the total group) conceals differences among subgroups. For example, can the aggregate statistic,10 percent incorrect invoices—be subdivided into sub-groups?
Definitely. One basis for subdivision is the two shifts doing the invoicing. For example:
Table One
| Shift | Invoice Error Rate | Number of Invoices Processed |
| 8 a.m.-4 p.m. | 0 percent | 1,000 |
| 4 p.m.-12 a.m. | 20 percent | 1,000 |
| 2,000 |
This subgrouping reveals a significant difference in invoice error rates. All incorrect Invoices were made during the second shift (20 percent). Since the number of invoices processed is equal, it is unnecessary to use a weighted average to arrive at the average Invoice processing error rate. In this case, we add the invoice error rates and divide by the total number of rates (0 percent + 20 percent /2 = 10 percent).
This Process Can Be Extended to Several Additional Subdivisions
By introducing additional subdivisions, and finding significant differences or variations among subgroups, we can move closer to determining the “root cause” of the incorrect invoices.
For example, the experience level of the employees working the two shifts supplies a possible explanation for the excessive number of the incorrect invoices. In this organization it was discovered that all new workers were routinely assigned to the second shift.
Table Two
| Experience Level | Invoice Error Rate | Number of Invoices |
| Experienced People | 0 percent | 500 |
| Inexperienced People | 40 percent | 500 |
| 1,000 |
Table Two indicates that new workers were responsible for all the incorrect invoices, if the subdivision process stopped at this point. It would appear that more intensive training of new billing staff could dramatically reduce the rate of incorrect invoices.
However, if we introduce one additional basis for subdivision, we learn more about the “root cause” of Incorrect invoices. What is the composition of the new invoice people on the second shift? That is, can the subgroup be subdivided on the basis of some other characteristic yet not under investigation?
Table Three indicates that inexperienced billing staff are of two kinds: 1) full-time employees and 2) part-time temporary staff.
Table Three
| Inexperienced People | Invoice Error Rate | Number of Invoices |
| Full-time Employees | 0 percent | 250 |
| Part-time Employees | 80 percent | 250 |
| 500 |
Table Three reveals that part-time temporaries are the cause of the incorrect invoices. Why? The full-time staff received thorough training in the invoicing policies and procedures.
But—and this is a very big "but"—the part-timers are brought in during peak shipping periods and are often thrown at the problem, receiving no training. That this happens shouldn't surprise you. However, it often gets overlooked when confronted with aggregated numbers.
People see what is presented to them; what is not presented tends to be disregarded. And what is presented, more often than not, are "problems couched in aggregate data"—especially in the areas where performance falls below expectations—which means that managers tend not to see the potential cause of the problem.
The Root Cause, Once identified, Can Be Eliminated via Management Action
How can the number of incorrect invoices be eliminated? Less dependence on part-timers in critical areas, and more effective training of the temporaries that are used. This particular organization stopped hiring temporaries for billing and invoice costs dropped eighty percent.
Management must change its policies. In this case, the hiring of office temporaries was eliminated. Management took an action to remedy the problem.
There was a corresponding savings in the customer support area. Customers no longer had to deal with chronically incorrect bills. (This example is a bit oversimplified but it is based on an actual case. Jim Harrington, a Deming missionary who worked at IBM for 35 years, estimates that 50 percent of the costs of every billing system are attributable to “screw ups." Why continue to pay that money?)
Now We Can Define Homogeneity
The critical question: Can the data be subdivided on the basis of some characteristic other than the one under investigation, so as to reveal differences among subgroups?
If it is Impossible to further sub-divide your data, the data can be classified as “homogeneous." If, however, one discovers a basis for subdividing which reveals differences, the data set is “heterogeneous” or “non-homogeneous.”
Aggregation has the effect of ignoring relevant subgroups. Conclusions that seem obvious when we look only at aggregated data become debatable when we examine "lurking variables" or relevant subgroups.
Homogeneity, Variability, and Statistical Procedures
Let’s review:
- An aggregated performance measurement is of limited diagnostic value.
- Through the process of isolating and analyzing variation among relevant subgroups, you can locate the “root cause” of the problem.
- Management action is required to deal with the “root cause” of the problem.
- Faulty conclusions and/or policies flow from a data-set that is not homogeneous with respect to the performance measurement under investigation.
Statistical procedures detect significant variation among subgroups. If significant differences in a performance characteristic (because of thoughtful subdivision of a data set) are found to exist, the reasons for the variation must be investigated. After the “causes” of the variation are discovered and eliminated, the performance measurement improves.
|
Contributor: From the Desk of the Executive Editor |
|
-
Explaining the Total Degrees of Freedom for Six Sigma Practitioners -
Is Your Continuous Improvement Organization a Profit Center? -
Kaizen: Rapid Change for a Rapidly Changing World -
Ratios: Abuses and Misuses! -
Preface To Performance Measurement: Understanding Ratios -
Turning Your Master Black Belt into a Strategic Process Improvement Leader -
Planning Process Improvement Project Strategy -
Applying an Engineering Process or Six Sigma to Your Sales Force -
Identifying 10 Potential Reasons for Transportation and Quality Problems -
Bringing Back the Romance for Lean Six Sigma
* = required.
Not a member yet? Sign up
You Should Check Out:
 |
Thank you for this very informative and well written article. I especially appreciate how clearly you illustrate the danger in how aggregate data can lead to faulty conclusions.
I am not clear how this is different from a mulit-variant study? |
Replies (0) |
Very interesting article.
In my opinion we need to better investigate the issues related on how to obtain the (right) classifications. In real life a high difficulty is to obtain data. In the example of the article we suppose we can get data related to
• the shift period where the Invoice error happened
• the level of experience of people that made the error
• the number of full-time people and part-time people
In real life we face two difficulties:
1. to realise which is the classification that match with the problem we are affording. In the example of the article we can have also different classifications that probably are not correlated with the Invoice errors, as sex of working people, age of working people, geographical area where they live, or days with nice or bad weather, etc.
2. to obtain the data that can be difficult per se and becomes even more difficult having in mind the previous problem (identify the right classification). The analysis of data with a certain classification is in fact an iteration activity and for each iteration we are asking new data “disturbing” with our request the operational organization that is currently working on other issues (in my opinion this is a reason why the quality people are seen as “disturbing” people)
In my opinion the point 1 is related to the capacity of observing the reality and then selecting the most interesting classification. The point 2 is related to IT organization and their ability to delivery easily data (for instance using data warehouse systems).
But of course these two points should be analyzed deeply. |
 |
Wonderfully clearly written! Thanks for writing this article. |
|
Such a concept of subdividing (or splitting the dcitionary) to focus on the root cause (or Red-X) is basis in Dorian Shainin approach (also called as Statistical Engineering). |
Advertisement
Events of Interest
-
Process Excellence Leaders Meeting
Holiday Inn Mart Plaza, Chicago, IL
September 13- 16, 2010 -
Business Process Excellence in Financial Services
Venue to be confirmed, Frankfurt, Germany
December 6- 8, 2010 -
Energy Process Excellence Network
Omni Hotel, Houston, Texas
December 6- 8, 2010 -
Business Process Excellence Congress
Venue to be confirmed, Johannesburg, South Africa
February 21- 24, 2011
You Might Also Like
-
The Legend of Motorola: Embracing the Next Generation of Process Improvement
In the 1980s, Motorola made waves in the business world with the development of its process improvement...Read more
Tina Huesing
-
Recipe for Success with Six Sigma in Healthcare
As I was waiting for my oil change, I flipped through a magazine and came across a photo of perfectly baked...Read more
Anantha Kollengode
Welcome to Six Sigma and Process Excellence IQ
|
Welcome to Six Sigma IQ! Subscribe now to our FREE bi-weekly Profit through Process newsletter and get exclusive access to training, insights and intelligence to enhance and strengthen your performance initiatives. |
|
|
|
Reach Six Sigma professionals through cost-effective marketing opportunities to deliver your message, position yourself as a thought leader, and introduce new products, techniques and strategies to the market.
Learn More >> Join today and interact with a vibrant network of six sigma professionals, keeping up to date with the six sigma industry by accessing our wealth of Six Sigma articles, videos, online seminars, live conferences and more. Sign Up Today! |
| Six Sigma IQ, a division of IQPC 2010 All rights reserved.Use of this site constitutes acceptance of our User Agreement and Privacy Policy. Careers With IQPC | Contact Us | Unsubscribe to Bizo |
|