Note that the Change Explorer is currently marked as a prototype as we are currently developing an alternative approach to the statistical testing that it employs.
The follwing table shows the Net Promoter Score over time from a survey of mobile phone owners:
The second period shows a significant decline in the NPS. The Change Explorer can be used to visualize the contributions to this change in terms of key subgroups. In this example we will look at the change in the NPS in terms of four key variables from the survey:
- Gender
- Age
- Main phone company
- Population density, or whether the respondents live a large cities, small cities, towns, or small towns / rural areas.
Within-group Averages
First we set the Contribution setting to Change in Average, which allows us to analyze changes in the metric of interest within each subgroup. The output is a bubble chart containing the key groups of interest with:
- The size of the group (within the second period) in the X-axis.
- The change in the average score for each group in the Y-Axis.
- The size of each group's contribution to the overall difference as the size of the bubbles. Negative contributions can be identified by hovering over the bubble, and the size of the bubble is proportional to the average value of a group's contribution.
- Whether the contribution for that group is statistically significant is indicicated by the color of the bubbles.
Hovering your mouse over each bubble will display the values for the group.
The output is either a table or a bubble chart containing the key groups of interest with:
From this chart we can see a number of things, including:
- The groups with the largest increase in NPS are those whose main phone company is Metro PCS and those who live in a Big city.
- The change for the Metro PCS is not statistically significant at the level we have chosen, but the change for the Big city is.
- The group with the largest decline is those whose main phone company is Verizon, and the change in this group is statistically significant.
Group Sizes
Next we set the Contribution setting to Change in Group Size. This allows us to analyze change in the metric of interest due to the relative increase or decrease in the size of the subgroups. For example, if a particular group always tends to give a low score, and this group increases in size between two periods of time, then that will have a negative contribution to the metric.
In this chart:
- The X-Axis shows the difference between each group's average score for NPS and the overall average for the first period. So groups on the left of the chart had a lower-than-average NPS, and groups on the right had a higher-than-average NPS in the first period.
- The Y-Axis shows the change in the size of the groups between the first and secon period. So groups on the upper half of the chart increased in relative size, and groups on the lower half decreased.
- The sizes of the bubbles indicate the contribution to the group size change, which is determined by multiplying the X and Y values. A group which increased in size, but which has a lower-than-average NPS is considered to have given a negative contribution overall. As above, you can tell when a group's contribution is negative by hovering over the bubble, and the size of the bubble is proportional to the absolute value of the contribution.
- The colors of the bubbles indicate whether the contribution has a significant decline or increase between the two periods.
From this chart we can see that:
- Respondents who have Sprint as their main phone company have a much lower-than-average NPS, and they grew slightly as a group between the two periods. This results in a negative contribution, but this is not significant at the level we have selected.
- T-Mobile had a higher-than-average NPS, and this group shrank by over 5%, which also results in a negative contribution. This contribution is marked as a significant decline.
- Respondents who live in small cities have a slightly-lower-than-average NPS, and the size of this group declined. This results in a positive contribution, and this contribtution has been marked as a significant increase.
- Respondents who have Verizon as their main phone company had a higher-than-average NPS and this group also increase, leading to this group being marked a a significant increase.
Average and Group Size
Finally, we set the Contribution setting to Change in Average + Group Size to analyze the total contribution of each group.
- The X-Axis shows the average score (NPS) for each group in Period 2.
- The Y-Axis shows the size of each group in Period 2.
- The sizes of the bubbles indicate the sums of the contributions for the change-in-average and change-in-group-size (as shown the previous two charts) for each group.
- The colors indicate the most significant result between the two contributions. So if a group was marked as No change for one chart, but as a Significant increase for the other chart, then they will be marked as Significant increase on this chart. See below for more information about statistical testing.
Options
Outcome The variable you wish to analyze. This should be Numeric.
Date A Date/Time variable to use to specify the periods to compare.
Profiling Variables One or more Nominal or Ordinal variables to use to profile the Outcome variable.
Contribution This controls which of the three analyses you wish to conduct. See above for examples.
Show contrasts as percentages Tick this to show the contributions (bubble sizes) as the relative proportion of contribution for each subgroup within the other subgroups from the same variable. This results in contributions which are divided by the sum of the absolute values of the contributions within each variable.
Output You can choose to display the Bubble Chart or a Table of the data used in the bubble chart.
PERIODS
This section allows you to control which two periods are used in the analysis.
Specify periods by You can choose to control the dates by Typing dates in YYYY-MM-DD format, or to choose dates via on-page Controls.
From/To/Label These settings allow you to choose the start and end dates for the two periods (periods always include both the start and end date) as well as a label for each period.
SIGNIFICANCE LEVELS
This section allows you to control the thresholds for statistical significance as z-scores.
Upper / Lower z-Score theshold Set threshholds for the significance levels as z-scores.
Label Customise the labels used for each significance level.
Color Choose a color to use in the bubble chart for this level.
Label when not significant Customize the label to show in the bubble chart legend for groups which are not significant at any of the entered levels.
Color when not significant Choose the color to use for data points for groups which are not significant at any of the entered levels.
Technical Details
Any cases which have missing data in the Outcome variable or the Date will be filtered out of the analysis.
The statistical testing used in these charts may give different results to that shown in Displayr's tables for small subgroups. This is because Displayr's default statistical settings are set to assume that groups have equal variance when the sample size is less than 10. To reconcile this difference you can select a table of interest, click Properties > APPEARANCE > Significance > Advanced and then change Test Type > Equal variance when sample size is less then to 2.
This tool is marked as a Prototype because we are currently investigating an alternative approach to statistical testing for to total contribution (that is, when the Contribtion setting is set to Change in Average + Group Size).
Comparisons of means use a t-test, and comparisons of group sizes use a z-test.
Next
How to Use the Change Explorer Prototype to Examine Change Over Time