How AI-Powered A/B Testing Works for Paid Traffic

We just rebuilt A/B testing across Leadpages. New engine, new builder integration, new dashboard. The reason we rebuilt instead of patching: traditional A/B testing wasn't built for the way marketing teams actually run paid traffic.

A landing page in 2026 gets traffic from search, social, email, retargeting, direct, and your existing list. Each of those audiences responds differently to different page variants. The classical approach (split traffic 50/50, wait for statistical significance, pick a winner, ship it to everyone) picks one variant that's the average best across all those audiences. It's almost never the best variant for any specific audience.

Leadpages now ships three layers of testing. You can use whichever fits your team and your traffic.

Layer 1: A/B testing on every plan

A/B testing is included on every Leadpages tier, starting at $99 a month. That's deliberate. Most landing page builders gate A/B testing behind a higher tier (Unbounce charges $149, Instapage charges $199). We don't, because if you can't run a test you can't improve a page, and we want every customer improving their pages from day one.

What's built into the page builder: an A|B toggle in the top toolbar that creates a variant. An experiment panel that slides in from the right with setup and results. Four goal types: form submission, click, purchase, and offsite conversion. A multi-element click picker for goals that fire on multiple targets. A traffic split slider. A declare-winner flow. Plain-English summaries of the results so you don't need a CRO specialist to read the dashboard.

There are also AI tools built into the editor. You can ask the AI to generate a new variant from a prompt, pull current test results in conversation, or suggest what to test next based on the page's content and your conversion goal.

Layer 2: Smart Traffic at Optimize ($199)

Once you've got two or more variants live, Smart Traffic is the next layer. It's a multi-armed bandit algorithm with an ML layer for visitor segmentation.

The mechanic: instead of splitting traffic 50/50 between variants and waiting for a winner, Smart Traffic routes each visitor to the variant most likely to convert for them. The signals it looks at: device, geography, UTM source, time of day, referrer. The algorithm is Thompson Sampling, with a configurable exploration/exploitation balance. The default is 80/20: 80% routes to the best-known variant, 20% explores to keep learning.

Variants that consistently underperform get auto-paused below a threshold so you're not burning paid traffic on losers. Manual override is always available if you want to pin a specific traffic distribution.

The dashboard shows three things: the AI's routing decisions, the estimated conversion lift versus a 50/50 split, and a "what-if" comparison so you can see what would have happened if you'd run a traditional A/B test against the same data. That last view is the one most marketers find the most useful. It makes the lift legible to stakeholders who didn't grow up on multi-armed bandit theory.

Layer 3: Auto-optimization at Scale ($399)

The third layer is for teams running a lot of variants on a lot of pages. Auto-optimization removes the "declare winner" step entirely. The system keeps optimizing the routing continuously based on incoming data, and learnings transfer across pages through cross-page learning.

This is the layer that approximates what enterprise CRO platforms like Optimizely do. The difference: we ship it inside the page builder at $399 instead of selling it as a separate experimentation suite at six figures.

When to use which layer

A/B testing if you've got a clear hypothesis you want to validate against a control. Pricing changes, big repositioning, fundamental UX shifts. The discipline of "I think X will beat Y because Z" is still where most of the actual learning happens for strategic decisions.

Smart Traffic if you've got two to five credible variants and paid traffic flowing from multiple sources. You're not trying to learn at this layer. You're trying to convert.

Auto-optimization if you're running enough volume that constantly resetting tests is a job in itself. The system runs continuously. You get the lift without the project management.

What it doesn't do

Leadpages A/B testing doesn't do server-side experimentation or feature flag testing. If you're testing rendered HTML on logged-in app pages, you want Optimizely Full Stack. If you're testing backend pricing logic, that's a different job.

Smart Traffic also has a cold-start problem. It needs traffic to learn from. If your page is doing 100 visitors a month, the algorithm doesn't have enough signal. Stick with traditional A/B until volume scales.

And the obvious one: AI routing optimizes within the variants you create. If every variant is bad, the system picks the least-bad. The art is still in writing variants worth testing.

Why three layers

The honest answer: a 5-person marketing team running a few campaigns wants A/B testing they can use without hiring a specialist. A 50-person team running hundreds of pages wants automation that doesn't require a dedicated CRO program. We could have shipped one and called it a day. We shipped all three because gating any of these behind enterprise pricing was something Unbounce and Optimizely were already doing badly, and we'd rather have every Leadpages customer running tests than gate the work.

If you want to see the testing layers running, the Leadpages free trial gets you the full product for 7 days. Build a page, ship two variants, watch the dashboard.