About this Course
This course will cover the design and analysis of A/B tests, also known as split tests, which are online experiments used to test potential improvements to a website or mobile application. Two versions of the website are shown to different users – usually the existing website and a potential change. Then, the results are analyzed to determine whether the change is an improvement worth launching.
This course will cover how to choose and characterize metrics to evaluate your experiments, how to design an experiment with enough statistical power, how to analyze the results and draw valid conclusions, and how to ensure that the the participants of your experiments are adequately protected.
Why Take This Course
A/B testing, or split testing, is used by companies like Google, Microsoft, Amazon, Ebay/Paypal, Netflix, and numerous others to decide which changes are worth launching. By using A/B tests to make decisions, you can base your decisions on actual data, rather than relying on intuition or HiPPO’s – the highest paid person’s opinion! Designing good A/B tests and drawing valid conclusions can be difficult.
You can almost never measure exactly what you want to know (such as whether the users are “happier” on one version of the site), so you need to find good proxies. You need sanity checks to make sure your experimental set-up isn’t flawed, and you need to use a variety of statistical techniques to make sure the results you’re seeing aren’t due to chance. This course will walk you through the entire process. At the end, you will be ready to help businesses small or large make crucial decisions that could significantly affect their future!