Not Working, or Too Early to Tell? How to Actually Know

Three months in. The posts go out, the product is live, the thing exists — and nothing has happened. You have exactly two stories available to explain it, and they cannot both be true. Either the approach is wrong and every week you keep going is a week thrown away, or the approach is fine and you simply haven't run it long enough for anything to surface. Most people stuck at this point think they have a patience problem, or a discipline problem, or a talent problem. They don't. They have a measurement problem: they never built a way to tell those two stories apart, so they pick whichever one matches their mood on the day they happen to ask.

You're not running a test. You're waiting.

A test has three parts, and all three have to exist before the thing starts, not after. A prediction — what specifically you expect to see if this works. A window — how much of the thing has to happen before you're allowed to read the result. And a reading — the moment you actually sit down, look at what happened, and say the answer out loud. Waiting has none of these. Waiting is doing the thing while an anxious background process asks, roughly once a day, whether it's been long enough yet.

That question — has it been long enough? — feels like the responsible one to ask. It's actually unanswerable, which is why it never stops getting asked. There's no amount of elapsed time that settles it, because elapsed time was never the variable that decides. You can post daily for six months in a format nobody was ever going to respond to, and you can post eleven times in a format that works and see it clearly on the eleventh. Weeks are not the unit. Weeks are just what your calendar happens to display, and because it's the only number visible without effort, it becomes the number you reason with.

What the waiting actually costs you

The obvious cost is the wrong call: you quit something that was about to work, or you hold something that was never going to. Both hurt, but both are survivable — you get another shot. The expensive cost is quieter, and it compounds. Because you never defined a prediction, you never got a reading. And because you never got a reading, you learned nothing. Which means attempt number two starts from exactly the same place attempt number one did, except you're more tired and you've spent three months of the only budget you actually have.

That's the mechanism behind a specific kind of person you've probably met, or been: someone who has worked genuinely hard for two years across five projects and cannot tell you one thing they know now that they didn't know then. Not because they weren't paying attention, but because they were switching on feeling rather than on findings. Every switch reset the clock and nothing carried forward. Effort accumulated; knowledge didn't. That's why the fifth attempt is as blind as the first, and why it feels so much heavier.

And there's the mirror-image failure, which is just as common and gets far more sympathy because we've collectively agreed that persistence is a virtue. You hold something for a year that a five-minute look at the mechanics would have killed on day three. Nobody applauds the quitter, so the sunk cost gets rebranded as commitment, and the longer it runs the more expensive it is to admit — which means the honest reading gets less likely with every week, exactly when it's most needed.

The eleven days we spent shipping something, and the day we deleted it

Here's a real one, from building Xpush. We had a browser extension. It wasn't a side experiment — we committed to it, folded it into the entry-level plan on July 21, and shipped it on August 3. On August 14 we deleted it. Not deprecated, not hidden behind a flag: the directory, the API routes, the credential store, the store listing assets, the line on the paywall card, the privacy clauses that existed only to describe it. Eleven days from shipped to gone.

Here's the part that matters. We did not delete it because the numbers were bad. We had barely any numbers — eleven days is nothing, and by every instinct I have about not quitting early, we should have let it run a quarter. We deleted it because of something that had nothing to do with elapsed time. Reading through the code that surfaced the inbound mentions inside it, I found a provider search that was being called without ever being metered — roughly three hundred provider credits a call, charged to us, billed to nobody — and the only thing throttling it was an in-process map that no serverless instance shares with any other instance. Which is to say: not a rate limiter at all. A hole with a shape that gets worse, not better, the more people use it.

That's a fact, not a vibe. And a fact of that shape has a property worth naming: waiting longer cannot change it. Another quarter of data would have told us more about whether people liked the extension, and precisely nothing about whether the cost hole closed on its own — it doesn't, it widens. Meanwhile there are pieces of this product we've held for months with far weaker evidence than that, and holding them was correct, because nothing we'd learned yet ruled them out. Same product, same month, opposite decisions. The variable was never how long it had been.

Quit on a fact, not on a calendar

This is the shift, and it changed how I decide things well outside of software. I no longer ask whether I've given something long enough. I ask what I would have to see for this to be working, whether I've seen it, and whether anything I've learned rules it out on structure rather than on results. Three questions with answers, in place of one question with none.

A calendar can only tell you how long you've been uncertain. Only a fact can tell you what to do about it.

The relief in this is real, and slightly surprising. Once a window is written down, the daily anxious check stops — not through willpower, but because the question genuinely isn't due yet. You're allowed to just do the reps. And when the window closes, the decision isn't an act of courage or a mood; it's reading a line you wrote yourself, back when you had no ego in the result.

The system: how to run a test you can actually read

1. Write the prediction before the window opens

One sentence, in a place you'll find again: if this is working, by the end of the window I will have seen ___. Write it before you start, because afterwards you will unconsciously edit it to match whatever happened. This single sentence is the difference between a test and a hope, and it takes about forty seconds.

2. Pick the fastest honest signal, not the one you want

Revenue is the signal you want. It's also slow, noisy, and downstream of five other things, which makes it a terrible instrument for a short window. Find the earliest thing in the chain that would still have to move: replies from people you'd actually want as customers, profile visits per thousand impressions, second messages from strangers, one person asking to pay before you offered. Honest means it can come back negative. If the signal you chose can't fail, you didn't choose a signal.

3. Set the window in reps, not in weeks

Not thirty days — thirty posts. Not a quarter — forty conversations. Reps are the unit the outcome actually depends on, and weeks are a proxy that breaks the moment life gets in the way. A month where you shipped four posts and a month where you shipped forty are the same month on the calendar and completely different experiments. Counting reps also fixes the most common misdiagnosis in this whole category: people who conclude the strategy failed when what actually happened is that the strategy barely ran.

4. Keep the two kill switches separate

There are exactly two legitimate reasons to stop early, and neither of them is discouragement. The first is a structural fact: something you learned about the mechanics that no additional data can reverse — the platform forbids it, the unit economics invert at scale, the thing you'd have to be good at is something you have no path to being good at. Structural facts kill immediately, on the day you find them, regardless of how the window was going. The second is the window closing on a missed prediction. Everything else — a bad week, a quiet Tuesday, someone else's launch making yours look small — is noise, and noise is not a kill switch.

5. Change one thing per window, and write down which

The universal move after a disappointing window is to change the topic, the format, the schedule, and the positioning all at once, which guarantees the next window is unreadable too. Change one. Note it next to the prediction. You'll be shocked how quickly a stack of one-line notes becomes the only genuine competitive knowledge you own — the stuff nobody can copy off you, because they'd have to have run your windows.

6. Read the result out loud, even when it's boring

The window closes and the honest answer is usually neither triumph nor disaster; it's something like "the prediction was half met and the reason is obvious in hindsight." Say it anyway, in a sentence, in writing. An unread test is a test you didn't run — you paid the full cost in time and collected none of the payout in knowledge, which is the worst available trade.

One next action

Take the one thing you're currently unsure about — the thing that made you search this question in the first place — and give it a prediction and a window in reps, today, before you do anything else with it. If that thing is your presence on X, then the rep is a post and the window is a count of them, and the reason people never get a readable answer there is that nobody remembers thirty posts well enough to read anything off them. That's the job Xpush does: it keeps the cadence so the reps actually happen, and keeps enough of your own history that the readout is a number rather than a feeling — measured against your own past posts, and deliberately showing an empty track until there are enough of them to say anything honest. Start the window at xpush.app, and stop asking whether it's been long enough.