Why We Ran Eight Rehearsals Instead of Two

On a recent Windchill upgrade we ran eight full rehearsals. The industry norm is one or two. On the face of it that looks like a team that needed more practice. It was the opposite, and the reason is worth understanding, because it changes how you should plan an upgrade.

Why the norm is one or two

Rehearsals are not limited by their usefulness. They are limited by their cost.

When a person executes every step of a dress rehearsal, each one consumes days of skilled time. So you plan for one, maybe two, and you try very hard to make them perfect. You front-load the analysis, you write the runbook carefully, and you hope the rehearsal confirms what you already believe.

That is not caution. It is what the price of a rehearsal forces you into. And it has a consequence people rarely name: with one or two attempts, you only find the problems you were already looking for.

What changes when the steps are automated

Once an AI agent is executing roughly 90% of the steps, the arithmetic inverts.

Wipe the system. Rehost from production down to the lower environment. Start again. What used to be a multi-day commitment becomes something you can run overnight, sometimes without being at the desk while it happens.

At that point quantity becomes the quality strategy. You stop trying to design the perfect rehearsal and start running rehearsals until the process stops surprising you.

Every rehearsal failed somewhere new

This is the part that matters, and it is the reason eight was not excessive.

Each run failed at a different point. Not the same failure eight times — eight different discoveries. And each one produced something durable: a rule of engagement, an entry in an error catalogue, an instruction for what to do when a specific condition appears.

The reason the failures kept moving is that a general-purpose model has enormous breadth and no proprietary depth. It has never seen your PLM environment. It does not know how a Windchill upgrade behaves, because that knowledge lives in the heads of people who have done twenty of them and in documentation that cannot possibly cover every scenario. So the domain expertise had to be supplied, error by error, until the gaps were closed.

By run seven and eight, there were no more surprises. That was the stopping condition — not a target number, but two consecutive clean runs with no human intervention.

What eight rehearsals bought

Three things, only one of which was expected.

Confidence on the weekend. Forty of forty phases executed on the production cutover. Nobody was nervous, which is a strange thing to be able to say about a two-hop upgrade on a clustered production system.

An environment that got cleaner. Running the process repeatedly surfaced accumulated configuration drift that had nothing to do with the upgrade — settings left over from older versions, misconfigurations nobody remembered introducing. Those were fixed before the production attempt rather than discovered during it.

A framework that carries. The rules, runbooks and error catalogue built over those eight runs do not expire when the project does. For a similar environment, we would expect three rehearsals or fewer. The eight was tuition, and it has been paid.

The honest caveat

This was a largely out-of-the-box environment. That matters.

A heavily customised, large, long-lived Windchill system is a different proposition. There you are also determining whether each customisation survives the version change, and whether existing configuration is actively blocking the upgrade. More rehearsals, and more human judgment inside each one.

The principle holds regardless: if you can make resetting a test environment cheap and fast, you can afford to rehearse until it is boring. And boring is the goal.