The End of Lazy Reporting

The Trending Delta

Making changes to your eCommerce site without measuring the impact is like getting dressed in the dark. The first step in getting smarter about these enhancements and understanding how moving needles relies on a commitment to measuring their effect. To get the most accurate measurement, we recommend testing every change you make side by side with the original. In the event that doing so isn’t possible, a before and after report is your next best choice.

The two most widely used types of before and after reports are “year-over-year” (YOY) and “period-over-period” (POP) techniques. While both give you a glimpse of improvement, they both leave room for error. Year-over-year doesn’t take into account trends you’ve already been seeing this year. You may have already been doing well or poorly. Meanwhile, period-over-period doesn’t take seasonality into account.

There is a third type of report. It’s not as simple as the two I just mentioned, and therefore not as popular. That said, it can give you a more accurate reading.

The Third Option

It’s called the trending delta, a statistical analysis that allows you to compare the trending data with observable data. Trending data is the result of several calculations, which take into account seasonality and the overall trend of your KPI. By comparing it to the observable data, we can calculate our improvement with increased confidence.

For example, the chart below displays the revenue of an eCommerce site over the past year. We’ll assume that a change took place between B and C.

Trending Delta Graph

We can see that comparing A to C (a YOY comparison) makes it look like the change made an improvement. The same goes for comparing B to C (a POP comparison). But when we compare D to C (a trending delta comparison), it’s pretty obvious that our change didn’t make an improvement at all. In fact, our change hurt revenue.

Trending Delta Real example

a real example of trending data for a business whose conversion rate was slowly decreasing

In the graphs above, trending data is shown as a very simple, linear progression but the truth is never so straight and simple. A trending analysis should take prior seasonality into account. For most retailers, this will primarily be expressed as high spikes in traffic and revenue at the end of the year. The more years you take into account, the more accurate your analysis becomes.

Trending delta real example of seasonality

a real example taking seasonality into account by comparing each month to the month before

Another important consideration when looking at trending data is how many changes you’ve made over the past year. The more changes you’ve made, the more erratic and less reliable trending data becomes. It’s like any experiment- introduce too many variables, and drawing a conclusion becomes fuzzy if not impossible.

Calculating Trending Data

There are a number of ways to calculate trending data. Your best bet is to export data from analytics into Excel and use a function to make your calculations. The first go will take some patience, but once you have it, the process shouldn’t take long at all.

Want to start calculating trending data? Check out this article from microsoft on how to use the FORECAST function in Excel to calculate trending data.

In Conclusion

If you’re going to test, measure. If you’re going to measure, be accurate. And if you aren’t going to use trending data, make sure you understand how POP and YOY data may be skewed. Lastly, remember that it’s natural to favor information that confirms your beliefs or hypotheses, but doing so will lead to making bad decisions.

by Founder & CEO
Kevin is the Founder & CEO of Blue Acorn, and he also serves on the Magento Certification Advisory Board. Kevin has worked with a number of companies with online needs: banks, IT companies, consumer packaged goods manufacturers, various software companies, and finally, starting his own eCommerce business. His roles included that of a web developer, a business development manager, a technology evangelist, and a consultant.  When Kevin founded Blue Acorn in 2007, he didn’t want to be all things to everyone who wanted to be on the web. As an eCommerce entrepreneur himself, he wanted to focus on helping other online retailers achieve their goals. At the time, the world of eCommerce “solutions” for online retailers was an overcrowded space of web development generalists that left many online retailers yearning for more – more expertise, more value, and more love. So Kevin started Blue Acorn and blogged about his thoughts and approach towards eCommerce. Those thoughts were well-received, and the demand for services to accompany them would soon follow.

DISCUSSION

2 Comments

  1. Finally! A big cure to me less productive days. Considering that there’s analysis and calculations involved in interpreting trending data in ecommerce, this will definitely give me a longer yet worth time to spend on. Thank you the share Kevin! This is so bookmark worthy! =)

  2. Leah Welsch

    Great post Kevin- informative and well written.

ADD YOUR COMMENT