← Engineering & Delivery Standards

Prepare to recover

Plan the recovery path before the change is urgent.

Not every change can be reversed completely, but consequential releases should identify failure signals, containment options, ownership, and a practical way to restore service or continue safely.

A reversible copper pathway with isolation gates and a stable return route across white marble

A conceptual representation of failure and rollback planning.

Why the business should care

Failure and rollback planning protects more than the technology.

Recovery decisions made during an incident are slower and more error-prone. A prepared response limits downtime, customer confusion, lost work, and uncontrolled secondary changes.

The business also needs to know what rollback cannot restore, such as information already sent, messages already delivered, or records changed by another system.

High-level method

How the principle is put into practice.

  1. Model credible failures

    Consider application, integration, credential, data, provider, network, user, and configuration failures in proportion to their consequences.

  2. Define stop signals

    Specify the observations that pause a rollout or disable a feature before damage spreads.

  3. Prepare containment and restore

    Preserve the last known good state, configuration, data backup, manual path, or feature control needed for recovery.

  4. Verify after recovery

    Confirm service, data consistency, unresolved work, notifications, and ownership rather than assuming a technical restart completed the recovery.

Illustrative scheduling integration

How the standard works in operation.

Situation

A new integration begins creating duplicate appointments after a provider-side change.

Approach

A release control disables new writes, preserves inbound requests, restores the prior booking path, and identifies records requiring reconciliation.

Business effect

The business continues operating while affected records are reviewed instead of allowing uncertain automation to continue.

What good work leaves behind

Evidence the business should be able to inspect.

  • Failure scenarios and operational impact
  • Rollback or continuity procedure
  • Backup and restore verification where applicable
  • Named incident and decision responsibilities
  • Reconciliation and follow-up record

The exact artifact may differ by project. The important point is that consequential claims and controls can be examined rather than merely asserted.

Common failure pattern

What to avoid.

A backup is not a recovery plan until the business knows what it contains, how it is restored, how long that takes, and what work remains afterward.

Connect the standard to the operation

Discuss what dependable delivery should mean for your business.

Begin with the operating need, the consequences of failure, and the evidence required before relying on the result.

Start a private conversation