Experimentation at Digitec Galaxus
How we evolved A/B testing from a product idea into a technical platform.
Loading article…
I'm based in Neustadt a.d. Weinstraße, Germany, and prefer remote work.
How we evolved A/B testing from a product idea into a technical platform.
At Digitec Galaxus, A/B testing was a central part of product development. In a large e-commerce shop there are constant questions where opinions alone are not enough:
Especially in a large shop with millions of visits and very different user groups, deciding purely by gut feeling is risky. A/B tests can inform product decisions when the technical foundation supports reliable measurement.
During my time at Digitec Galaxus, I worked on the internal experimentation platform and helped evolve it over several iterations. This was not just a small frontend feature, but a system deeply integrated with routing, server-side rendering (SSR), UI, backend, analytics and observability.
The most important lesson for me: Good A/B tests are not just an if in the code. They need a smart architecture concept, clean measurability, good developer experience and clear ownership.
We developed an experimentation platform across backend and frontend that allowed teams to serve and measure variants reliably without hurting performance.
The core elements:
Math.random()The first simple version of A/B testing in the shop was extremely pragmatic. On request, the system essentially rolled the dice to decide which variant a user would see:
const variant = Math.random() < 0.5 ? 'A' : 'B'
That is enough to validate a concept rudimentarily. For real production experiments, it is unusable. Assignment is neither stable nor traceable, and users risk seeing a completely different UI on the next click.
We needed something deterministic.
A naive approach to stable A/B testing would be setting one cookie per running experiment: exp_checkout=A, exp_button=B. With dozens of simultaneous experiments this massively bloats request headers and makes state management painful.
We chose a more elegant path: deterministic hashing based on a single ID.
Instead of storing variants, we stored and communicated only one stable visitorId. This ID was passed through the whole stack, from frontend to the deepest backend service.
Whenever a system needed to know which variant a user belonged to, it calculated it itself:
function getVariant(visitorId: string, experimentName: string): 'A' | 'B' {
// Generates a deterministic hash from ID and experiment
const input = `${visitorId}:${experimentName}`
const bucket = hash(input) % 100
return bucket < 50 ? 'A' : 'B'
}
The big advantage: No external tool had to be queried. Server-side routing, React components and backend recommendation services could calculate the variant synchronously from the visitorId in context, without a network request.
Tech aside: why hashing is so powerful
A hash turns input into a seemingly random but fully deterministic value. The same combination of user ID and experiment name always produces the same bucket. Variant distribution is mathematically even, without storing state for concrete experiments.
The theory with the visitor ID was great, but in practice, server-side rendering (SSR) in a Next.js architecture exposed a hard timing problem.
To track a user consistently, the first visit needs to assign an ID and store it as a cookie (Set-Cookie).
The problem: During exactly that first page request, the SSR server already has to fetch data from internal APIs to render the HTML. If we only put the newly generated ID into the Set-Cookie header for the browser, internal APIs used during SSR would not know about it yet. The user would receive default data (variant A) on the first request, and only on the second request, when the browser sends the cookie, variant B would appear.
To solve this, we had to adjust the state flow:
visitorId as random bytes.Set-Cookie header, but also injected into the internal request context immediately.Set-Cookie.This guaranteed that even highly dynamic, data-driven checkout or recommendation experiments rendered correctly from the first frame, without layout shifts.
A/B tests are only useful if teams can build and debug them efficiently.
As a developer or PO, you want to test the feature in the browser. But because assignment is deterministic, you might simply be in the wrong variant. Manually editing your cookie is awkward with hashed values.
We therefore built an internal helper tool into the UI. It allowed employees to define: "For experiment X, I want to be in variant B."
Under the hood, the solution was pragmatic and clever: Because we knew the hashing implementation, the tool simply generated new visitorIds very quickly in the background until it randomly found one whose hash for this specific experiment produced exactly "variant B". That ID was then stored as a cookie. All other active experiments remained "normally" randomized for this test user, which created an extremely realistic test environment.
The best experiment is useless if the data is wrong. We separated evaluation into two clear areas:
It often happened that one variant converted better but was technically less stable. Combining both worlds directly was essential.
Especially important in this context were A/A tests. These are experiments where both groups see exactly the same thing. If our analytics dashboards suddenly showed significance, we knew our tracking, hashing or reporting had a bias. Plausibility checks were the key to trusting the data.
Over time, we kept evolving the platform. Beyond simple UI toggles and checkout flows, we also used the system for real infrastructure tests.
For example, we tested image delivery dynamically: different providers, different formats (WebP vs. AVIF) or new caching strategies. All of this could be controlled deterministically via the propagated ID, down to the CDN or image service.
As an engineer, this meant you could pitch an architecture decision, implement a test variant and prove with real LCP values from RUM analysis that the idea was actually better.
A tool does not scale by itself when 10 teams deploy code at the same time. Experiments are temporary code, and without process this quickly turns into real technical debt.
We needed fixed rules:
My role often shifted: I helped evolve the system architecturally, but also coached teams, formulated technical hypotheses and made sure unclear product ideas became cleanly measurable code variants.
The platform gave teams a shared way to assign variants, run experiments and investigate the results.