500,000 Rows That Look Real: Load Testing Without Uniform Random Junk

A load test that passes is not automatically good news. Quite often it means the data was too tidy to expose anything.

This is the failure I see most in performance work. Someone generates a million rows, runs the suite, sees acceptable latency, and signs off. Then the feature ships and falls over at a tenth of that volume, and everyone is baffled because the numbers were fine.

The numbers were fine. The distribution was wrong.

Uniform data is the wrong shape

If you generate orders by picking a random user for each one, every user ends up with roughly the same number of orders. With a million orders and ten thousand users, everyone has about a hundred, plus or minus a bit of noise.

Real systems do not look like that at all. A small number of accounts generate most of the activity. A long tail generates almost none. Some accounts signed up and never came back.

That difference is not cosmetic, because most performance problems are specifically problems with the heavy end.

Your customer detail page loads a user's orders. Under uniform data every page load fetches about a hundred rows and everything is fast. Under real data, most pages fetch a handful and one fetches forty thousand. The average looks identical. The experience does not, and the timeout only ever happens to your largest customer, who is also the one you least want to annoy.

What uniformity hides specifically

Missing indexes look unnecessary. With even distribution a sequential scan on a moderate table is tolerable. Add skew and the same query on the heavy account becomes the thing that saturates the database.

N+1 queries stay quiet. A loop that issues one query per related row is fine at ten rows. It is a catastrophe at ten thousand. Uniform data never gives you the ten thousand case.

Pagination bugs hide. Deep offset pagination degrades badly, but only past a page count that uniform data never reaches for any single entity.

The planner makes different choices. Postgres picks plans using statistics about value distribution. Feed it uniform data and it makes reasonable decisions that turn out to be wrong once real skew arrives. You are not just testing with the wrong data, you are testing a different query plan than the one production will use.

Cache hit rates lie. Real access patterns are concentrated, so real caches perform better than uniform ones in some places and much worse in others. Either way your measurement is not transferable.

Volume is the easy half

Getting to half a million rows is not hard. Any loop can do that.

The hard part is making those rows have the right shape:

A few entities holding a disproportionate share of the children. Dates clustered around business hours and weekdays, not spread evenly across the calendar. Text lengths that vary the way human input varies, with a few outliers long enough to matter. Nullable columns that are actually null a realistic proportion of the time, because a code path that never sees a null is a code path you have not tested. Status fields weighted the way real workflows weight them, mostly completed, some pending, a few in the states that only happen when something went wrong.

None of that is exotic. It just does not happen by accident, and random() will not give it to you.

Size the data to the question

One more thing worth saying, because it cuts the work down.

People reach for the largest number they can generate, on the theory that bigger is more rigorous. Usually the useful question is narrower. You are not asking "does the system work at a million rows". You are asking "does this endpoint stay under 200ms when one account has 40,000 orders".

That second question needs a specific shape far more than it needs a big total. Half a million well shaped rows will teach you more than ten million uniform ones, and it will finish generating today.

Decide what you are trying to break before you decide how much data to make.

Where SeedQL fits

SeedQL generates data from your schema with referential integrity intact, at volumes up to 500,000 rows per generation on the Business tier, and pushes it into your target database. Larger runs go through an async job so a long push does not block everything else.

Now the part where I have to argue against my own product, because this post spends a thousand words saying uniform data is useless and it would be dishonest to skip it.

SeedQL does not currently generate skew. Foreign key values are chosen uniformly from the available parent keys. So if your load test hinges specifically on one account holding 40,000 of the 500,000 orders, SeedQL will not produce that shape for you today. Weighted distributions are a known gap and they are on the roadmap, and when they ship they will be announced in the changelog rather than quietly added to a feature list.

What it does give you is the part that is tedious rather than clever: a schema filled correctly at volume, with foreign keys intact, up to 500,000 rows per generation on the Business tier, pushed into your target database. Runs are seeded, so the same seed reproduces the same dataset exactly, which matters when a load test finds something and you need to run it again.

If you need a specific hotspot today, the practical route is to generate the bulk with SeedQL and then concentrate rows onto a chosen parent yourself with an UPDATE. That is a few lines of SQL against a database that is already structurally valid, which is a much smaller job than building the whole dataset by hand.

The check worth running

Before your next load test, run one query against your test dataset. Count the children per parent, order descending, look at the top ten.

If the top row is close to the median row, your data is uniform, and whatever the test tells you afterwards is about a system you do not operate.