Load testing versus real-device testing

"Is it fast?" is two questions wearing one coat. The first is whether your infrastructure keeps answering when many people arrive at once. The second is whether the thing that arrives on someone's phone is usable. Load testing answers the first with real rigour and cannot answer the second at all. Understanding where the boundary sits saves a lot of wasted effort — and a lot of incidents where every dashboard is green and users are still complaining.

What load testing actually measures

Tools in the k6, JMeter, Locust and Artillery family generate synthetic traffic against your endpoints and report what the server does under it. They are excellent at this, and nothing else replaces them.

• Throughput: how many requests per second the system sustains before it degrades.
• Latency distribution: median, 95th and 99th percentile response times under a given concurrency.
• Error behaviour under saturation: where timeouts, connection resets and 5xx responses begin.
• Capacity planning signals: how the system scales as instances, pool sizes or cache hit rates change.
• Regression detection on the backend: whether a release made an endpoint slower under load.

These are infrastructure questions and load testing is the correct instrument for them. If you are launching a campaign, planning a sale event or sizing a cluster, this is the work.

What load testing structurally cannot see

A load generator is not a browser. It sends requests and reads responses. Everything that happens after the response arrives — the part the user experiences — is invisible to it.

• JavaScript parse, compile and execution cost on the user's processor.
• Layout, style recalculation and paint on the user's device.
• Compositing and animation smoothness, and whether frames are dropped.
• Memory pressure and garbage-collection pauses on constrained hardware.
• Third-party scripts loaded by the page in a real browser, which a request generator never fetches.
• Perceived milestones: when the largest element appears, when the page first responds to a tap.

What real-device measurement adds

Device-side measurement inverts the question: given hardware someone actually owns, how much capability is there, and what does your page do with it? That produces a different set of numbers.

• Processor throughput on the main thread, which predicts how long your scripts will block input.
• Memory bandwidth, which predicts collection pauses and image decoding cost.
• Layout and DOM cost, which predicts scroll jank and interaction stutter.
• Compositing capacity, which predicts whether your animations hold their frame rate.
• Network latency and jitter from where the device is, rather than from a data centre.

Together these describe the performance floor: the least capable device you are willing to support, and what it can afford. Budgets written against that floor survive contact with real users. Budgets written against a developer laptop do not.

Where each belongs in a workflow

Use load testing when

• You are sizing infrastructure or planning for a traffic event.
• You need to know where the system breaks and how it fails.
• You are validating autoscaling, connection pooling, rate limits or queue behaviour.
• A backend release needs a regression gate under concurrency.

Use real-device measurement when

• Users report slowness the team cannot reproduce.
• You need to choose a minimum supported device and defend the choice with numbers.
• You are deciding whether an animation, a framework or a widget is affordable on your floor device.
• You want a frontend regression gate that reflects hardware rather than a synthetic lab score.
• Your analytics show high bounce on mobile with healthy server metrics — a classic signature of a device-side problem.

And where lab audits fit

Single-page audit tools run one page in one controlled browser and produce a score with suggestions. They are useful as a checklist and as a shared vocabulary. Their limitation is the controlled part: the environment is a simulation, usually a throttled desktop pretending to be a phone. It approximates a device class; it does not measure the device in someone's hand, and it says nothing about behaviour under concurrency.

The three instruments answer three questions and overlap far less than their marketing suggests: load tools tell you whether the service survives, audit tools tell you whether the page is built sensibly, and device measurement tells you whether the hardware your users own can run what you shipped.

A combined method that works

• Establish the device floor first. Measure the cheapest device class you support and record its subsystem scores.
• Write frontend budgets against that floor: payload ceilings, response-time ceilings and a cap on third-party origins.
• Enforce those budgets in continuous integration so regressions fail the build rather than surfacing in support tickets.
• Run load tests on the backend against realistic traffic shapes, not uniform request floods, before events that change traffic volume.
• Collect field data from production to check both models against reality, because real users will always find conditions neither test invented.

If you are choosing an approach right now, the practical starting point is to measure the hardware, since that is what constrains everything else. Take a reading, then use how to read a device performance score to interpret it, or see Redline compared with k6 for a direct side-by-side.