A/B Testing
Tell Seba what change you want to test and it returns a complete A/B test plan — a sharp hypothesis, the one variable to vary, the sample size, the metrics, and the rule for calling a winner. Billed per run.
A test that produces a real answer, not a guess
A hypothesis, not a hunch
Seba frames the test as “because [data], we believe [change] will cause [outcome] for [audience]” — a specific prediction you can actually prove or disprove.
Sample size up front
It calculates how many users per variant you need from your baseline and target lift, so you don’t peek, stop early, and ship a false positive.
Metrics that matter
One primary metric tied to business value, secondary metrics for context, and guardrail metrics so a “win” doesn’t quietly break something else.
Just describe the change
No spreadsheet, no calculator. Say what you want to test in Telegram and the plan comes back as a file, billed per run.
What the test plan covers
The same rigor a senior experimentation lead applies, in one pass.
- Hypothesis — a specific, data-grounded prediction, not “let’s see what happens”
- One variable — a single, meaningful change so you know what actually worked
- Test type — A/B, A/B/n, MVT or split-URL, matched to your traffic
- Sample size — users per variant from baseline rate and target lift
- Primary metric — the one number tied to business value that calls the test
- Secondary & guardrail metrics — context, plus things that must not get worse
- Traffic allocation — 50/50, conservative or ramping, with consistency rules
- Analysis plan — significance, effect size, segments, and how to read the result
- Documentation — hypothesis, variants, result and learning for your playbook
From idea to test plan in three steps
Open Seba in Telegram
One tap into @meetseba_bot. No install, no signup forms.
Describe the change to test
“Test a new pricing headline” plus your traffic and current rate. That’s the brief.
Get the plan
Seba returns a full test plan — hypothesis, sample size, metrics and the rule for calling a winner — with the exact credit cost.
When to plan an A/B test
Decide with data instead of opinion.
- +Comparing two headlines, CTAs or layouts
- +Pricing-page or checkout changes worth getting right
- +Onboarding or signup-flow tweaks
- +Email subject lines and send variations
- +Settling a “which version is better” debate
- +Building a steady experimentation program
Frequently asked questions
What does the A/B test plan include?+
A specific hypothesis, the single variable to test, the test type, required sample size per variant, primary / secondary / guardrail metrics, traffic split, and an analysis plan for calling the result.
How does Seba calculate sample size?+
From your baseline conversion rate and the smallest lift worth detecting. Tell Seba your current rate and traffic and it returns users-per-variant so you can pre-commit before launching.
Why test only one thing?+
Change several things at once and you can’t tell what moved the metric. Seba isolates a single meaningful variable per test so the result is interpretable — and suggests an A/B/n or MVT setup when you truly need more.
Does Seba run the test for me?+
Seba designs the test and tells you how to read it. You implement the variants in your tool (PostHog, Optimizely, VWO and similar); ask Seba to write variant copy or an analysis checklist and it will.
How much does a test plan cost?+
You spend credits per run, and Seba shows the exact cost before it starts. Start free with welcome credits; credits never expire.
Do I need any testing tools?+
No tools to plan the test. Seba builds the plan inside Telegram and returns it to the chat; you only need a testing tool when you’re ready to ship the variants.
Plan your next test in a single message
Tell Seba what you want to test and get a statistically sound plan back. Pay only for the run.
No install · no signup forms