Somebody already decided which questions your business can ask
It was probably a developer, three years ago, and nobody reviewed it.
A client asked me what looked like a simple question.
They had switched off their on-site experimentation platform, conversion had fallen, and they wanted to know what the switch-off had cost them. A recommendation to drop the platform permanently was already being drafted.
Four things had changed inside eight days. A sale ended. There was a payment incident. A release went out. And then, last of the four, the platform was switched off.
By the time anyone looked, more than 40% of the conversion decline had already happened before the platform was touched.
Checkout completion had collapsed two days earlier, on the day of the release, and had not recovered since. Engagement was flat throughout, either side of the switch.
Removing on-site experiences changes what visitors see, and a change in what visitors see shows up in engagement. A sale ending does not. It changes whether people buy.
So the read-out was almost certainly measuring the sale, and possibly the release, and attributing both to the platform.
To settle it properly you need four things: order mix between orders containing discounted items and orders containing none, cart abandonment, session duration by journey, and page depth by journey.
Not one of them existed.
No item-level discount field on any ecommerce event, on any journey, so discounted and full-price orders cannot be separated. No cart view event anywhere in the funnel, because the basket page emits no ecommerce data at all. Session duration and page depth collected perfectly well sitewide, but page type values inconsistent across templates, so neither can be cut by journey.
A business was days away from making a permanent platform decision on evidence that could not support one. And not because anything was broken. Every tag was firing. Consent was gated correctly. The QA pass had come back clean.
The choice nobody remembers making
The cause sat upstream of every tool in that conversation, and it was not really a technical
failure.
At some point somebody wrote a page template and chose which fields it would emit. That choice got made on a Thursday, under delivery pressure, by a developer who had never been told what the business would need to ask.
Nobody reviewed it, because it never failed. Reports kept loading. Numbers kept appearing.
And it quietly fixed the boundary of what that company is able to know about its own trading.
No amount of tag manager work rescues it either. A container can rename, cast, enrich, suppress and route, but every one of those operations acts on data the page has already pushed.
It cannot carry a field the page never emitted.
It cannot construct a discount that does not exist in the source object.
It cannot fire a basket event where no basket data was pushed.
So a dataLayer is not a tracking artefact. It is a specification of the questions your business is permitted to ask. In most estates it was written by accident, by people who were never told what it was for, and it has not since been read by anyone in a commercial role.
Which is worth being blunt about. The failure does not sit with the developer, who did the job they were given, to the brief they were given. It sits with the fact that nobody on the commercial side ever wrote down what the business would need to know, and nobody has looked at the specification since.
If you are accountable for trading performance and you have never seen your own
dataLayer, that is not a technical oversight. It is the most consequential document about your business that you have never read.
The cost of that is not theoretical. Every promotion you cannot evaluate is margin given
away without knowing whether it bought anything, and you will run the same promotion again next quarter because you have no evidence not to.
Every media platform bidding without a pre-purchase value signal is optimising towards the wrong customers with real money. Price the two together across a year, against the cost of adding fields to a template, and the ratio is not close.
Once you see it, it is everywhere
That estate was not unusual. The same failure keeps turning up in different clothing: a field the page never emitted, and a question that therefore cannot be asked.
No item-level discount or coupon. Whether a promotion drove incremental orders, or simply discounted orders that would have happened anyway.
A basket page that emits no ecommerce data. How many customers built a basket and left, what was in it, what it was worth. Abandonment audiences have nothing to populate from either.
Product prices with no top-level value. What any step before purchase is worth. Value-based bidding gets no pre-purchase signal, and no release can be judged in money.
No payment attempt or outcome recorded. Whether customers are being declined or choosing to leave. With no attempt there is no denominator, so a failure rate cannot exist even in principle.
Price sent as text on one event and as a number on another, same page, same product. Warehouse queries fail or silently drop rows, and dynamic remarketing cannot read a price that arrives as a string.
A list identifier that is really the list's display name. Rename a merchandising list and its history splits in two.
Page and site type values that differ between templates. Engagement is collected perfectly well and cannot be cut by journey, which is the most frustrating category of all, because the data is sitting right there.
The one I almost never find is stock status at the point of view. In stock, low stock, out of stock, per item, at the moment the customer looked at it. Without it, when conversion falls on a category, demand and availability are indistinguishable.
Merchandising knows exactly which sizes were out last Tuesday. They have always known. It simply never reached the event. Did we lose the customer, or did we lose their size?
Nobody chose any of this. That is the whole point. It was never specified, so each template solved it locally, and every local answer was perfectly reasonable at the time.
Why it survives for years
Two reasons, and they compound.
Nothing about it fails loudly. Broken tracking gets escalated within a day, because somebody notices a chart go flat. A missing field is never escalated at all. There is no error, no alert, and nothing turns red.
The question gets asked once, receives a vague answer, and quietly stops being asked. The business adapts to not knowing and calls that normal.
It always loses the queue. In the case above, the development team was at capacity on a replatform, so all measurement work had been deferred behind it. Which is the exact inversion of the right answer, because the templates that emit the dataLayer were the very things being rewritten.
Writing a correct push into a template somebody is already authoring costs a fraction of retrofitting one that has been built, tested and signed off. The busiest moment is the cheapest moment, and it is almost always the one that gets skipped.
The order almost everyone gets backwards
The fix is a sequence, not a tool.
Start with the decisions your business makes on a cycle: promotional planning, range and buying, media allocation, platform investment. For each decision, write down the questions it turns on. For each question, list the fields required to answer it at event level. Then design the schema. Then build the tags.
Done in that order, the dataLayer is a contract between the business and the site, and a commercial director can read it, because it is written in questions rather than parameters. Done the usual way, from the tool backwards, you get an estate that answers whatever the tool suggested and none of what the business asks.
It needs one named owner who is not the person shipping the feature, and it needs to sit in the definition of done for every template change. Without both it never ships, because it is nobody's feature and it has no launch date.
Back to the decision
That client did not drop the platform on the strength of that read-out. Not because the data eventually answered the question, but because the sequence of events was enough to show that it could not.
That is where most businesses actually sit. Not measuring, but reasoning around a gap and hoping the reasoning holds. It usually does. Right up until the decision is expensive, or the person doing the reasoning is not in the room.
The uncomfortable part is that fixing it does not fix the past. Collection is forward-only and there is no backfill. An event that was never sent cannot be recovered, at any price, by anyone. The gap you leave open this quarter is a hole in next year's evidence, and it will be there when somebody senior finally asks.
Which is why the honest answer to a question your data cannot answer today is not "give us two weeks".
It is "ask again in a year".
The Five Question Test
Free, 45 minutes, and you keep the output whether or not we go any further.
Bring the five questions your leadership asks most often about trading performance. The real ones, in the words they actually use.
I will take each one live and work backwards: the question, the fields required to answer it at event level, and whether your dataLayer is emitting them today. You leave with a written list of which of the five you can answer, which you cannot, and what each of the gaps would take to close.
No deck, no proposal. If your estate turns out to be in good shape, I will tell you so and we will be finished in half an hour.
I can run a few of these a month around client work.
I am Marc Alexander. I run Metric Owl, an independent analytics consultancy in London. I specify measurement for ecommerce businesses: what the pages must emit, in what order, and which commercial questions that makes answerable.
Comments