How to measure your push notification effectiveness, here is a simple and streamlined method for measuring the effectiveness of your push notifications. One thing to mention is the difference between “effectiveness” and “success”. Effectiveness is a change in behavior and success is a result. For example, if a push notification campaign that was sent to all customers resulted in 30% of all customers making an order, then your campaign was effective.

However, effectiveness requires context, if the week before your new campaign 30% of customers already made an order, then you have zero effect because that 30% still would have made an order. Now that we got that clarified, here is how to measure push notification's effectiveness in 4 steps.

What is the expected result?

Before you start any measurement, you are going to need to know what you are measuring. Thus determining what you intend to achieve with your push notification campaign, more orders, more signups, more items added to the cart, more orders of item XYZ. This will go along with determining the context, you will need to identify which customers are already fitting your mold.

Determine the context

As I mentioned earlier, you need to establish a baseline. You can establish a baseline simply by taking a time period directly before the campaign, for example, 1 or 2 weeks, and calculating the results. If you wanted a more sophisticated measurement of your effectiveness, you could determine possible reasons for your baseline. Why did 30% of customers order without “being asked”? How is this 30% different from the other 70%? Now if you are trying to create a new metric, ie introducing a new product, then an option to best define context would be to create an estimation. X percentage of customers order XYZ and thus more probable to order new XYZ.

Control groups and putting into action

This step is optional and falls into how sophisticated you want to get. Since you already have your context, you would measure if you were to target a percentage of that context group versus the entire population(customer base) do you see more positive results. This will also play more into what your results mean than what they are. But if you just want simplicity, then comparing the before and after will suffice. If you aim for a more holistic view, ie. If you want more X-type customers to do XYZ then group tests would be a good option.

The results

Now look at the results, did you see a positive or negative difference in results. A good rule of thumb in statistics is to use something called a P-value to determine uniqueness. A P-value is a good method to determine if your push notification campaign actually made a difference or the difference in results is just caused by random chance. P-values can get pretty complicated, if you are interested you can check out this resource here.

But for simplicity's sake, you can make your P-value .05, the industry standard, meaning the difference in results is greater or less than 5%, and if so then you are sure that your push notification campaign had an effect. In the end, measuring the effectiveness of a push notification campaign is not difficult and is a better metric to watch out for rather than overall results.