Opticon 2015: In a Nutshell

Opticon 2015

Recently our team spent several days in San Francisco with our partners from Optimizely at Opticon, their annual conference. Between exciting Optimizely announcements, spending time with clients and partners, and attending information sessions, it was a successful trip to the west coast.


Dan Siroker, Co-Founder & CEO of Optimizely, took the stage to make several announcements. The crowd was really buzzing over the new Optimizely Personalization tool. It is in private beta now, with general availability coming this fall.

How is this new tool different from Optimizely’s widely-adopted optimization platform? Personalization essentially allows one to target segments of visitors based on behavior (for example, visited product page “A” two times) or demographics to offer them personalized experiences. On the other hand, conversion testing focuses on determining which changes, or combination of changes, result in optimal on-site performance.

At Blue Acorn, we believe that the best approach is to begin with a conversion testing strategy to ensure that your site is functioning to the best of its ability. Then, once the site is in order, layer a personalization strategy on top of it to really hit it home.

Even though the intention of personalization isn’t to “test” per se, please note that Optimizely will initially be holding back 5% of any given segment to create a baseline for performance comparisons. The remaining 95% will be exposed to the personalized experience.

While visiting with clients and partners at our booth is always a good time, I was able to sneak away on a few occasions to attend several of the information sessions organized by Optimizely. As a conversion optimization consultant, I use the platform on a daily basis. I get deep into the weeds, seeing both the limitations and the vast possibilities of conversion testing, so I am always on the lookout for opportunities to build on my knowledge of existing and new capabilities.

Here’s a quick recap of three sessions I attended.

Pushing the Boundaries of Optimizely: This session was super development heavy. While development isn’t my forte, I did bring home notes for my colleagues. My favorite line from the session was: “Javascript, the answer to and cause of all life’s problems.” Now, that I can relate to.

Testing with Stats Engine: Why it’s Harder to Find Winners with Optimizely: Prior to Optimizely’s Stats Engine, an analyst was responsible for determining whether a sample size was large enough for results to be considered relevant. With Stats Engine, this is no longer necessary. The biggest takeaway that they emphasized on several occasions was the importance of setting a primary goal for every test. The primary goal, above all secondary goals recorded through Optimizely, receives a greater amount of statistical power behind its calculation. Setting the primary goal prior to launch is key to obtaining optimal (and hopefully more timely) results for your test’s primary KPI. Check out this Optiverse article to learn more about setting a primary goal.

Early Lessons in Personalization Strategy: I am quite excited to start working with Optimizely’s new personalization tool. One feature that really stood out to me was the Audience Explorer. As an alternative to digging through site analytics to determine optimal segments to test, Optimizely provides suggestions for which segments to target using predictive analytics. Panelists who shared their own stories recommended using a combination of data and gut instinct to get started with a personalization strategy. Fair enough.

Beyond all of the inspiring optimization talk, the view from Opticon’s location at Pier 27 in San Francisco wasn’t too shabby either. Thanks for a lovely week, Optimizely!

San Francisco Bay

About Shana Braun

Shana has worked in the field of data management and analytics for a decade, and she actually enjoys it. She got her start in the “big data” side of business where she worked in data management for large CPG companies. She went on to spend a few years with an advertising agency reporting on all digital activities including eCommerce analysis. As a Conversion Consultant at Blue Acorn, Shana now works with clients to create a data driven approach and client specific strategies to increase key eCommerce performance metrics.


  • Brian Lang says:

    Great summary! I’m bummed I wasn’t there to enjoy the event 🙁

    On this point: “The primary goal, above all secondary goals recorded through Optimizely, receives a greater amount of statistical power behind its calculation.” it is worth mentioning that this is at the cost of increasing your chance of accepting a false positive – this a result of Optimizely not adjusting for the multiple comparison problem on your primary goal.

  • Shana Braun says:

    Hi Brian, thanks for the note. I appreciate the add and hope you’re able to make it to Opticon next year!

Leave a Reply

Your email address will not be published.