Deckmetric
    Jump to letterABCDEFGHIJKLMNOPQRSTUVWXYZ
    Product & PMF
    Entry
    Global · Global

    A/B Test

    Also called: AB test, Split test, A/B testing

    TL;DR

    A controlled experiment that compares two versions of a feature, page, or flow to determine which produces a better outcome.

    An A/B test randomly splits users into a control group (A) and a variant group (B), exposes each to a different experience, and measures the impact on a chosen metric. With enough sample size and a clean random assignment, the difference between the two groups can be attributed to the variant.

    Doing it rigorously requires pre-registering the hypothesis, picking a primary metric, computing the required sample size in advance, and not peeking at the result early. Most teams underpower their tests and over-trust early signals.

    Worked example

    A pricing-page A/B test routes 50% of visitors to a $49/mo price and 50% to $79/mo. After 4 weeks and 8,400 visitors per arm, the $79 variant converts 1.4% vs 1.7% for $49, but ARPU is 36% higher, so the $79 variant ships with 95% statistical confidence on revenue per visitor.

    Common pitfalls

    • •Calling a winner before reaching statistical significance.
    • •Running too many simultaneous tests on the same surface and corrupting attribution.
    • •Optimizing local UI changes while ignoring the larger funnel.

    When this shows up in a pitch deck

    Rarely a slide on its own, but A/B test discipline is implied when growth metrics improve substantially over a known baseline.

    Related terms

    Funnel Analysis

    Decomposing a user journey into ordered steps and measuring conversion between each step to find the biggest drop-off.

    Conversion Rate

    The percentage of users who complete a desired action, sign-up, purchase, upgrade, out of those who had the chance to.

    Activation Rate

    The percentage of new sign-ups who reach the product's defined aha moment within a target time window.

    Lean Startup

    A methodology for building startups under uncertainty using rapid Build-Measure-Learn cycles instead of long product plans.

    Cohort Analysis

    Grouping users by sign-up period and tracking each group's behavior over time to spot trends invisible in aggregate metrics.

    Use A/B Test in your next pitch deck

    Deckmetric scores your pitch across 10 VC frameworks and against 8 investor types.