You Can’t Prove ROI You Never Baselined

The first question I ask on a new engagement: what metrics do you track today?

The most common answer is a room of blank stares.

Not defensiveness. Not evasion. Genuine uncertainty — because nobody has ever asked them to think about it that way, and whatever data exists is scattered across systems that were never queried for this purpose.

I’ve stopped finding it surprising. But it’s worth understanding what it costs you, because the cost arrives about two years later and by then it’s unrecoverable.

The trap

Here’s the sequence that plays out over and over.

A company commits to a transformation. Somebody builds the business case, usually around the one metric the organization has always used — most often headcount reduction, because that’s how every project has been justified for twenty years.

The program goes live. The promised savings don’t materialize in that specific column. The project gets quietly filed as a disappointment.

And then the actual benefits show up — eighteen months, two years later — in a column nobody was watching. Change churn drops. Rework falls. Engineering capacity frees up. Scrap declines.

Except nobody can prove any of it, because there’s no “before” to compare against.

The win happened. It just isn’t provable. And in most organizations, a benefit you can’t demonstrate to the people who approve budgets did not occur.

Why the returns land somewhere unexpected

This isn’t bad forecasting. It’s structural.

Transformation programs change how information moves through a company. The second- and third-order effects of that are genuinely hard to predict, and they’re frequently larger than the first-order ones you built the case on.

The clearest example I know: a company that couldn’t prove a single head saved, then discovered two years later that annual engineering changes on a mature product line had gone from about 450 to 40. Nobody forecast that. It happened because manufacturing joined the design review and quality started getting built in rather than caught downstream. Full story here.

The returns show up somewhere other than where you pointed the camera. Plan accordingly.

Check out the video below to hear how this plays out in real life:

What to capture before you start

You don’t need a measurement program. You need a snapshot, and rough beats nothing.

Change and rework

• Engineering changes per year, by product line

• What share originate from defects caught downstream

• Average change cycle time

• Scrap and non-conformance rates

Speed

• Time from design release to first article

• Time from eBOM to a usable mBOM

• New product time-to-market

Effort

• Share of engineering hours on sustaining vs. new product

• Hours spent manually reconciling data between systems

Quality and service

• Warranty claims traceable to a design or process issue

• Field failure rates

• Time to answer “what configuration is unit 7?”

If you can only get three: engineering changes per year, share of engineering time on sustaining work, and how long an eBOM-to-mBOM handoff takes today. Those three catch most of what actually moves.

“Our data is too messy to baseline”

I hear this constantly, and it’s usually true. It’s also not a reason to skip it.

A rough number captured today beats a precise number you can’t reconstruct in two years. If your change data is approximate, write down the approximation and note how you got it. Consistency of method matters far more than precision — you’re measuring a delta, not certifying a figure.

And the exercise itself has value independent of the baseline. Asking “how many changes did we write last year and why?” surfaces things. Usually uncomfortable things. That’s useful before you start, not after.

Don’t commit to a single metric

If your business case rests on one number, you’ve created a binary outcome for something that will produce a spread of results.

Better approach: name a primary metric, then explicitly list secondary ones you’ll also check. Say out loud in the business case that returns in this category of work frequently appear in unexpected areas, and that you’ll be measuring more broadly than you’re committing to.

That single sentence buys you the room to claim a win you didn’t forecast — which, based on what I’ve seen, is the win you’re most likely to get.

Then actually go back and look

This is the step everyone skips.

The benefits that take two years to appear require somebody to go looking two years later. By then the program is over, the team has moved on, and nobody owns the question.

Put it on a calendar. Assign it to a person. Because when you do go back and find the number, you get something more valuable than the number itself: proof, to the people who fund this work, that the last hard thing you asked the organization to do actually paid off.

When people are in the trenches of a transformation, they cannot see wins. They see disruption. Being able to come back later and say here’s what you did, here’s the number, this is yours — that’s what keeps the next initiative funded. Without it, every future proposal starts from zero.

The short version

• Most manufacturers can’t say what they currently measure.

• Business cases get built on one familiar metric, usually headcount.

• Real returns often appear elsewhere, 18–24 months later.

• With no baseline, those wins are real but unprovable — which is functionally the same as not happening.

• Capture a rough snapshot now: change volume, sustaining vs. new-product effort, handoff cycle time.

• Name a primary metric but commit to checking several.

• Schedule the look-back, and give it an owner.

If you’re planning a PLM or digital-thread program right now, the highest-value hour you’ll spend this month is writing down what your current state looks like. You will want it later, and you cannot go back and get it.

Element Consulting has completed 127 PLM implementations. We start engagements by asking what you measure today. Related reading: 400 change orders a year, down to 40 and the engineering–manufacturing handoff. Talk to a Windchill expert →