A/B Testing for Data Analysts

A/B testing is one of the highest-leverage skills a data analyst can have: it's the difference between telling a company what happened and telling them what to do next. This course takes you from the foundations of experimentation through statistics, hypothesis testing, and experiment design, into the core statistical tests analysts use daily, honest result interpretation, and advanced pitfalls like Simpson's Paradox and multiple testing. You'll close with hands-on SQL and Python workflows and two full portfolio projects, so you finish with real work to show, not just notes.

Rutvik Acharya

Principal Data Scientist

Atlassian

Loading modules...

What You'll Learn

Design statistically sound A/B tests, including sample size, randomization units, and experiment duration

Run and interpret the right statistical test for any metric: t-tests, z-tests, chi-square, and Mann-Whitney U

Apply advanced concepts like sequential testing, Bayesian analysis, and multi-armed bandits

Build a full analysis workflow in SQL and Python, from raw event logs to a stakeholder-ready report

Who Should Attend

Data analysts who want to move beyond dashboards into experimentation and causal reasoning

Marketing and growth analysts running tests on campaigns, pricing, and conversion funnels

Product analysts and product managers who need to design and interpret A/B tests Aspiring data scientists building a portfolio with real, defensible statistical projects

Anyone preparing for data analyst interviews that include A/B testing and statistics questions

CERTIFICATION

Certificate of Completion

9 Modules

11 Hours 30 Minutes

43 Lessons

43 Challenges

Language: English

FAQ

FREQUENTLY ASKED QUESTIONS

No. The course starts with a full statistics refresher covering populations, sampling, distributions, probability, and confidence intervals before it introduces hypothesis testing. If you can work with basic math and are comfortable learning some new terminology, you can start from the beginning.

Basic familiarity with SQL and Python is helpful but not required to start. The course introduces SQL query patterns and Python code using scipy.stats and statsmodels step by step in Module 8, with all earlier statistical concepts explained independently of any specific tool.

You will be able to design a statistically sound A/B test, choose the correct statistical test for a given metric, calculate sample size and Minimum Detectable Effect, correctly interpret p-values and confidence intervals, and build a complete analysis workflow in SQL and Python from raw data to a stakeholder-ready report.

Yes. Module 9 includes two complete, end-to-end portfolio projects built on realistic datasets: an e-commerce checkout conversion test and a mobile app onboarding retention test. Each project includes a business problem, a full dataset, guided questions, and a complete worked statistical solution.

Yes. Module 7 covers advanced considerations including Sample Ratio Mismatch detection, network effects and interference between users, sequential testing, Bayesian A/B testing, and multi-armed bandits, alongside the trade-offs of each approach compared to a standard fixed-horizon test.

Every concept is taught through a single running business scenario, NorthLoop Coffee Co., so you see how sampling, hypothesis testing, and experiment design connect to real product and business decisions rather than abstract examples. The course also emphasizes common mistakes and misinterpretations that are usually left out of purely theoretical material.

The course covers the two-sample t-test, the z-test for proportions, the chi-square test for categorical outcomes, and the Mann-Whitney U test for small or skewed samples, along with a clear framework in Module 5 for choosing the right test based on your metric and data.

The course is organized into 9 modules and roughly 11 to 12 hours of material in total, including reading, diagrams, code walkthroughs, and practice exercises. You can move through it at your own pace.

Yes. Every lesson includes interview-style questions covering the concept just taught, and the two portfolio projects in Module 9 are structured so you can walk an interviewer through a complete, defensible analysis from hypothesis to recommendation.

Yes. A Certificate of Completion is issued once you finish all 9 modules and both portfolio projects.

Scroll Up

GET FREE CAREER COUNSELING