Testing Framework - Predictive/Statistical

Independent testing includes majorly 5 components as depicted above. Each of the 5 components are further explained below taking the industry frameworks like PMBOK, ITIL, Six Sigma, Lean, Agile, CMMI Services etc.,

Similarly Usage or application of statistics can be grouped into 4 major categories as below.

Below diagram depicts few of the tools/techniques used in each of the above 4 categories.

Below slides shows few of the examples in each of the above categories with the alignment to the testing scenarios/processes/metrics.

Below slides provides few more examples/models that can be applied in actual testing i.e., test case generation, test execution, test planning etc.,

Improve Phase - Identify the Solutions

Heat Map - Visual Representation

Heat map is one of the useful and powerful data-analysis tools available in business intelligence. Heat Map is visual representation of data using colors instead of numbers only. This tool is used to analyze the complex data sets for a quick and easy way of understanding.

Popular heat maps being referred/used in the industry includes

• Election results by geography
• Visitors interaction with a webpage
• Usability or consumer experience

There are many ways to create heat maps but the common understanding in all of these representation is usage of colors to communicate the numbers and their relationships. Heat Maps are mostly used for two dimensional representation. But the advanced heat maps can be drawn for more than two dimensions. For example cell size and color both can be used to represent a different relationships. One can add sliders to filter/zoom the data and its relationships as required by the user. If you have to represent the same situation using a bar chart, the visual would be cluttered and difficult to understand.

Example:

Below heat map used to understand the customer feedback from a service operations. Customer provided his feedback with the services offered by the vendor. Feedback can be positive (Happy with the service provided) or negative (Not happy with the services). Purpose of this heat map is to understand the customer view of the services provided with appropriate filters and drill-downs.

Quality Function Deployment

•       Gather Voice of Customer (VOC)
•         Rate the requirements on a scale of 1-5
•         Understand competitors ratings with respect to ours
•         Identify/map technical parameters that influences requirements
•         Build the correlation among requirements & technical parameters.
•         Also relation among technical parameters
•         Analyze the matrix to understand
•         Complete the QFD Matrix

Sources of Variation

Sources of variation can be categorized into two major categories. These are called common cause variation and special cause variation.

Common Cause Variation:

·        Present all the times in the process

·        Individually will have a minor effect

·        Collectively the variation can be added up leading to significant effect

Special Cause Variation:

·        Not always present in the process

·        Appear sporadically

·        Come from outside the process

·        Can have large or small affect on variation but typically have major impact

Strategy to address special cause:

·        Gather the data real time to signal the special causes quickly

·        Take immediate actions to reduce the damage

·        Investigate for the cause – Understand what is different

·        Plan for a long term solution

Strategies to address the common causes involves

·        Stratify – Identify the patterns in the way the data is clustered or do not clustered

·        Experiment – Make planned changes and learn from the effect

·        Disaggregate – Break the processes into small pieces and manage the pieces effectively

7 Step method of process improvement:

1.      Purpose

·         What are we trying to do?

·        What problem/gap is being addressed?

·        What is the impact?

·        What are the other reasons to fix this gap?

·        How to know things are better once improved?

·        What is your plan for this project?

2.      Current Situation

·        What is the history?

·        What are the symptoms of the problem?

·        Where do they appear?

·        What happens when the problem occurs?

·        Who is involved?

·        Can we draw the flowchart to depict the process?

3.      Cause Analysis

·        What are the causes for the symptoms?

·        Which can be verified using data?

·        What are the potential root causes?

4.      Solutions

·        What actions will address the root causes?

·        What criteria can be used to compare the solutions?

·        What are the pros and cons of each solution?

·        Which is the best? Which one will be selected?

·        How to test in a small scale? How to verify with data?

·        Which solution proved to be more effective?

·        What are the plans for full scale implementation?

5.      Results

·        How will the results meet the targets?

·        How well the plan executed?

·        How the results can be sustained in future?

6.      Standardization

·        What is the new standard method?

·        How will the users be trained?

·        What is in place to ensure the results are maintained?

·        How the results will be monitored?

·        What means are in place to foster ongoing improvements?

7.      Future plans

·        What is not addressed by this project?

·        What are the recommendations?

·        What is being learned from the project?

·        How the documentation will be finished?

·        What is the exit criterion to close the project?

·        Did we meet the exit criteria?

Six Sigma - Levels of Root Cause Analysis

It may not be necessary to spend equal efforts for any type of problem/issues. The kind of investigation may vary depending on the issue frequency/impact/nature. This articles details out the multiple level of Root Cause Analysis (RCA) that can be approached deepening on the problem nature.

Six Sigma – Root Cause Analysis (RCA)

Introduction: Beneath every problem there is an underlying cause but we need to identify the root cause to prevent the recurrence. Many times the actions taken to address the issue reoccur again in the same place or in a different place. This symptom indicates that the root cause is not been identified/addressed. Root Cause Analysis (RCA) is a structured approach for identifying and eliminating the underlying root causes. It may not be feasible or necessary to conduct RCA for all the issues as it involves time and effort. RCA is an analytical tool to perform a comprehensive, system-based review of critical incidents. Primary objectives of RCA includes

·        The primary aim is to identify the root cause(s) and prevent that problem from ever recurring

·        Systematic way of approaching and resolving the problem

·        Prevent the recurrence at lowest cost in the simplest way

Steps:  

1. Define the problem
2. Gather the information
3. Plan for Root Cause Analysis
4. Conduct RCA
5. Develop the solutions and action plans
6. Manage the Action items

Six Sigma – Root Cause Analysis Techniques

Many techniques available in conducting Root Cause Analysis, each technique has its own advantages and suitable for a particular situation. Common, widely accepted and simple to use techniques includes

·        Cause & Effect diagram

·        5-Why Analysis

·        Brainstorming

3 

 

Cause & Effect Diagram: Also called as Fishbone diagram. This is a tool for identifying all the causes of an effect. The effect being examined is the problem/opportunity that has to be eliminated. C&E Diagram is a graphical representation of the causes and effect. Use this technique for

·        Multiple causes has to be grouped logically

·        Understand width and depth of the causes

·        Problem is repetitive

Steps:

·        Write down the effect to be investigated and draw the backbone arrow (as below)

Note: KB Article – Knowledge Base Article has to be selected by the engineer to resolve an issue reported by the customer. Selecting of wrong article either delays the resolution or unnecessary escalation to a next level as the issue is unresolved by the engineer. 

·     Brainstorm with the identified people considering all the broad areas/ groupings of the potential causes of the effect “Classified KB article is incorrect”. Rule of thumb consider the generic 4 categories i.e., Man, Method, Material and Environment. You can define your own categories and may use affinity diagram to group the cause.

·        Group the causes identified during the brainstorming into logical groupings to represent the Cause & Effect relationship.

·        Drill Down all the causes for further reasons and goon extending the branches till the root cause is identified (May use 5-Why technique as required)

Five – Why Analysis: 5-Why is a problem solving technique that allows us to reach the root cause by repeatedly asking the questions. Even though this technique is called “5-Why” we may reach the root cause with fewer or more than five questions. Use this technique

·        Repetitive issue without any supporting data

·        Simple and low risk problems that does not require significant analysis

·        Problem is very specific to a process/ system (Not spread to multiple processes)

Steps:

·        Define the problem

·        Gather the team and confirm the problem

·        Ask the first Question Why? Record all the answers on a whiteboard or flipchart

·        Ask few more successive “Why” until we reach no further causes

·        Confirm the root cause and proceed for next set of actions

Example: Customer complaints on the delay in Pizza delivery

·        Why there is a delay in the Pizza delivery?

o   Delivery boy not reached on time

·        Why the delivery boy not reached on time?

o   He could not find the address

·        Why he could not find the address?

o   Address given to him is incorrect

·        Why the address is incorrect?

o   Address is not available in the records

·        Why the address is not available in the records?

o   Customer is new, manually note down the address

·        Why the address is incorrect?

o   Incorrectly noted while taking orders

Solution: Incase of the first time users, confirm the address once again and provide the telephone numbers to the delivery boy to reach out to the customer in case of any issues.

Brainstorming: One of the very widely used and easy to use techniques for analyzing the problem to reach the root causes. Brainstorming generates ideas and later evaluated to finalize and confirm the causes.  

Steps:

·        Establish a clear objective, Re-phrase for confirmation

·        Create a list of questions

·        cover all potential causes in the following four areas People, Process, Environment, Tools

·        Document all the findings and agree on the same 

Analytics – Lead Time Prediction

Below approach can be used to predict the lead time of completing a project considering the variation and cycle time distribution from the past data. The results are interpreted considering the probability of interest.

Steps to predict the Lead Time:

• Collect the data from past related to cycle time and data may include the sub divisions i.e., cycle time may include the actual cycle time, hold time, cycle time of multiple steps etc.,

o   E.g., Project of setting up a new server may include purchase, installation steps. While gathering the data we can collect the data at the desired level to capture the uncertainty in each steps while predicting.

  

•        Understand the distribution of the data

•     Calculate the Lead cycle time for multiple probabilities considering the distribution data is following. For this example we are considering that the data is following Normal Distribution. We can use Excel Function NORM.INV (probability, mean, standard_dev) to calculate the Lead time (Predicted) for the project considering different probabilities

• Another way of predicting the lead cycle time is through simulation. Assuming normal distribution generate the normal variables

o   Cycle Time : Mean (31) and Standard deviation (17)

o   Hold Time : Mean (23) and Standard deviation (24)

o   Add up to get the Total Cycle time = Cycle Time + Hold Time

• From this you can interpret that 90% probability cycle time = 103 days
• Probability to complete the project in 60 days = 55%