Lucia Van Den Brink has run nearly 1,000 experiments in 14 years.

She's worked with 100M+ brands, founded The Initial to help companies build experimentation in-house, co-founded Women in Experimentation, and teaches at CXL.

I invited her on Love at First Try to talk about something most SaaS teams get wrong: how to actually make data-driven decisions without overcomplicating it.

This episode is for you if you've ever thought "we don't have enough traffic to test" or "A/B testing is too expensive for us." Lucia challenges both of those assumptions.

🧠 What you'll learn in this episode:

0:00 - Intro: what this podcast is about and who it's for
0:25 - Who is Lucia and why experimentation matters for smaller SaaS teams
2:59 - The real difference between random testing and building an experimentation culture
6:36 - Why experimentation is actually about scaling leadership (insight from Booking.com)
9:15 - What counts as an "experiment" beyond simple A/B tests
11:32 - How a news website validated a 6-month feature before building it
12:57 - Why starting with removing elements is one of the biggest growth levers
16:22 - How to validate your data before trusting any experiment
19:05 - How to prioritize what to test (and where to start)
21:34 - Why you shouldn't segment your tests when you're just starting
25:41 - How to measure the right KPIs (including delayed metrics)
31:28 - Why you should never measure just one metric
36:45 - Real example: reducing churn in the first two weeks with a get started page
43:10 - Why "obvious UX improvements" still need testing (the humbling 20-30% win rate)
48:09 - The biggest mindset shift from junior to senior in experimentation

💡 Actionable insights from Lucia:

1. Start by removing, not adding
   One of the biggest growth levers is removing elements from your pages. Most tools let you hide elements without code. Try removing a field, a section, or a step in your funnel. You'd be surprised how often less friction means more conversions.
   
   

2. Run an AA test before any real experiment
   Before you trust your data, run an empty test (control vs. control). This tells you if your tracking actually works. Skip this and you might be making decisions on broken data.
   
   

3. Measure multiple KPIs, not just one
   Pick a main metric, but always track 2-3 supporting metrics. If you're testing onboarding changes, measure signups AND activation AND delayed metrics like paid conversions. One number never tells the full story.
   
   

4. Consider negative testing
   Instead of building a big new feature to test, try removing the opposite. Want to know if onboarding calls help? Test what happens when you remove them. You learn faster and cheaper.
   
   

5. Calculate the business case for "losing" metrics
   Sometimes a test hurts one metric but helps another. If showing a phone number increases support calls but also increases conversions, do the math. The revenue might cover the cost.

___

Introduction

Jim Zarkadas (00:00)
Hey, I’m Jim, and this is the Love at First Try podcast — a podcast for SaaS CEOs and developers who want to learn more about design without it feeling overly complex.

In every episode, we explore how to design products that become sticky and unforgettable — covering taste, UX, growth, and practical frameworks you can apply.

Let’s dive in.

Guest Intro: Building In-House Experimentation

Jim Zarkadas (00:25)
We always start with a quick intro — who you are, what you’re working on, and who you help.

Lucia Van Den Brink (00:47)

I’m Lucia, founder of The Initial.

We help companies build in-house experimentation programs — instead of outsourcing testing.

Recently, we’ve focused on large companies (€100M+ revenue), because for them experimentation should be a core competitive advantage, not an external service.

I also run a community called Women in Experimentation and share content on LinkedIn.

What Experimentation Really Means

Jim Zarkadas (02:59)
If I’m a SaaS founder doing ~€1M ARR with no real experimentation experience — where should I start?

Lucia Van Den Brink (03:09)

Experimentation is about systemizing decision-making.

Instead of guessing:

• You look at data

• Understand customer behavior

• Form hypotheses

• Test changes

For smaller companies, it’s not about reducing risk — it’s about finding growth levers.

There’s usually a lot of low-hanging fruit.

Experimentation Culture vs Random Testing

Jim Zarkadas (05:19)
What separates companies with real experimentation culture?

Lucia Van Den Brink (06:36)

It’s about scaling decision-making beyond founders.

Early-stage companies rely on intuition.

But as you grow:

• More people make decisions

• Less direct customer understanding

• More complexity

Experimentation replaces intuition with a repeatable system to learn what works.

The biggest value isn’t just revenue — it’s learning.

What Counts as an Experiment?

Jim Zarkadas (08:21)
Is experimentation just A/B testing small UI changes?

Lucia Van Den Brink (09:15)

Not at all.

Experiments can include:

• Research

• User interviews

• Feature validation

• A/B tests

A/B testing is just the highest-confidence method, not the only one.

Example:

A news company validated a major feature (auto-refresh feed) through:

1. Research

2. Prototyping

3. Testing variations

4. Final A/B test

Only then they built it.

The Biggest Myth: Experimentation Is Expensive

Jim Zarkadas (12:27)
Is experimentation expensive?

Lucia Van Den Brink (12:57)

It can be — but it doesn’t have to be.

You can start very simply:

• Use free tools

• Run 1–2 experiments/month

• Remove elements instead of building new ones

Removing elements is one of the highest-impact tactics.

Even non-technical founders can start.

Tools Don’t Matter as Much as You Think

Jim Zarkadas (15:04)
Any tool recommendations?

Lucia Van Den Brink (17:16)

Tools don’t matter that much.

The only thing that matters:

👉 Can it split traffic correctly?

Many teams get stuck choosing tools — and never start.

That’s the real problem.

How to Decide What to Test

Lucia Van Den Brink (19:48)

Use simple prioritization:

1. Start with high-impact pages (not homepage)

2. Focus on:

   • Product pages

   • Funnel steps

3. Prioritize:

   • Above-the-fold changes

   • High-traffic areas

Don’t overcomplicate it early.

Metrics: The Biggest Mistake Founders Make

Jim Zarkadas (29:59)
What should you measure in experiments?

Lucia Van Den Brink (31:28)

Never rely on a single metric.

Use:

1. Primary KPI

(e.g. trials, conversions)

2. Supporting metrics

(e.g. activation, engagement)

3. Delayed metrics

(e.g. revenue, retention)

You need a holistic view.

Example Insight: More Friction Can Be Better

Lucia Van Den Brink (14:14)

Removing friction isn’t always good.

Example:

• Removing a field might increase conversions

• But reduce lead quality

Sometimes adding friction improves outcomes.

Experimentation Mindset Shift

Jim Zarkadas (43:10)
What about ideas that feel like “obvious UX improvements”?

Lucia Van Den Brink (43:10)

Most ideas don’t work.

Typical outcomes:

• 20–30% wins

• 20–30% losses

• Rest = no impact

So “this will definitely work” is usually wrong.

Experimentation is humbling.

Why Not Testing Is Actually Riskier

Lucia Van Den Brink (46:00)

Not testing feels faster — but it’s riskier.

If you implement something wrong:

• You harm conversions

• You don’t know why

• Fixing it takes longer

👉 Testing is often the safer and faster path.

Experimentation = Organizational Power

Lucia Van Den Brink (48:56)

The biggest shift from junior → senior thinking:

It’s not about winning tests.

It’s about:

• Learning what works

• Making better decisions

• Empowering teams

Experimentation gives anyone in the company a voice — backed by data.

Even an intern can drive impact.

Closing

Jim Zarkadas (51:40)
This was a great conversation — thanks for joining.

Lucia Van Den Brink
Thanks for having me.