Selasa, 02 Februari 2010

Significance of Statistically Significant Results in A/B Testing

Significance of Statistically Significant Results in A/B Testing - Hi, friend Internet Marketing, in this article entitled Significance of Statistically Significant Results in A/B Testing, we have prepared this article well and concise to be easy to understand for you to read and can be taken inside information. hopefully the contents of the post Article A/B Testing, Article optimization, that we write this you can understand and useful. okay, happy reading.

You are running an A/B test (or multivariate test) but are in a hurry to make a decision to pick the winning page. Should you pick a winner based on initial few days of the data before your tool has actually declared a clear winner? This question comes up quite often during the conversations with the clients.

I always warn against such an approach and advice to be patient and get statistically significant results before pulling a plug on an underperforming variation o declaring a winner. I wanted to share few graphs with you to illustrate my point.

The following example is from a test that I am currently running.
First Two Weeks: If you make a decision without completely getting through the test, you will pick “Yellow” as the winner while declaring “Blue” a loser.


Month Later: There does not seem to be a clear winner.


Few More Days Later: Seems like “Blue” is trending higher. Should you pick “Blue” now?


Finally: We don’t have a clear winner yet.


As you can see, picking a winner or dropping a loser in the early stages of test, without having a statistical significant result, would have been a wrong decision.

Comments? Questions?


Looking to fill your Web Analytics or Online Marketing position?

Post your open jobs on Web Analytics Job Board
--------------------------------------------------------------------------



Thank You and Good article Significance of Statistically Significant Results in A/B Testing this time, hopefully can benefit for you all. see you in other article postings.

You are now reading the articleSignificance of Statistically Significant Results in A/B Testing with the link address http://www.ir2.me/2010/02/significance-of-statistically.html

0 komentar

Posting Komentar