Contract Testing and Version Compatibility_

This article is part of the series From Code to Production: A Quality Framework for Multi-Service Platforms. The hub explains the whole framework. This article goes deep on one question: how do you know a change to one service will not break another service that is already running?

It uses the series’ running example. The Orders API accepts orders and calls Payments over gRPC. Orders also publishes OrderPlaced events, which Inventory and Notifications consume. This article covers the synchronous boundary between Orders and Payments, and the public HTTP API that Orders exposes to clients. Event contracts have their own article, Testing Async Messaging and Schema Evolution.

Series status. The other deep-dive articles referenced below are still being written. Their titles appear in italics, without links, until they are published.

The failure

The Payments team decides that amount_cents should be a decimal amount so it can support currencies with three decimal places. In payments.proto, they add a new field and delete the old one:

// Before
message ChargeResponse {
  string charge_id = 1;
  ChargeStatus status = 2;
  int64 amount_cents = 3;
}
 
// After
message ChargeResponse {
  string charge_id = 1;
  ChargeStatus status = 2;
  reserved 3;
  reserved "amount_cents";
  string amount = 7; // decimal, for example "25.99"
}

The change follows the protobuf rules: the deleted field number is reserved so it can never be reused.¹ The Payments tests pass. The generated code compiles. Payments deploys.

Orders is still running the previous version. It reads field 3, which Payments no longer sends. Protobuf does not raise an error for a missing field. It returns the default value, which for an integer is zero.

Nothing crashes. Every order is recorded as free until someone notices in the finance reports.

Two things let this through:

1. Orders tested against a mock. Its test double for Payments still returned amount_cents, because the mock encoded what Orders believed Payments does.
2. Nobody tested the version combination that ran in production. Payments v9 was tested with its own suite. Orders v42 was tested against a mock. The pair Orders v42 plus Payments v9 was never checked before it existed in production.

Contract testing addresses the first problem. Version-aware deployment gates address the second. You need both.

What a contract is

A contract is an explicit, versioned, machine-checkable statement of how two services interact. It captures what one side sends, what the other side returns, and the conditions under which that holds.

There are three common ways to produce and check one.

Consumer-driven contracts were described by Ian Robinson in 2006. The key idea is that a provider should know exactly which parts of its interface consumers rely on, so it can evolve everything else freely.² Bi-directional testing is a PactFlow feature, not part of open-source Pact. It compares a consumer contract with a provider’s OpenAPI document statically, and never replays the consumer contract against provider code.³

These approaches complement each other. In the example platform, the Orders-to-Payments boundary uses consumer-driven contracts, and the public Orders HTTP API uses a provider-owned OpenAPI spec with breaking-change detection.

Consumer-driven contracts in practice

The rest of this section uses Pact, the most widely used open-source consumer-driven contract tool. Pact supports gRPC and protobuf through its protobuf plugin, which runs a gRPC mock server built from the service’s .proto file.⁴

The workflow

1. Orders writes a consumer test. The test runs the real Orders client code against a Pact mock of Payments. The mock records what Orders sent and what it needed back.
2. The test produces a contract, which Orders publishes to the broker, tagged with its git commit and branch.
3. The broker triggers Payments’ verification, through a webhook when the contract changes, and also as part of every Payments build.
4. Payments replays the contract against its real code and publishes the result, with its own version and branch.
5. The broker now knows which Orders versions work with which Payments versions. That knowledge drives the deployment gate.

Writing the consumer test

The consumer test exercises Orders’ real payment client. The following is an abridged sketch using Pact JS with the protobuf plugin. The exact API varies between Pact and plugin versions, so check the plugin documentation for your setup.

const pact = new PactV4({ consumer: "orders-api", provider: "payments" });
 
const contents = {
  "pact:proto": path.resolve("proto/payments.proto"),
  "pact:proto-service": "Payments/Charge",
  "pact:content-type": "application/protobuf",
  request: {
    order_id: "matching(type, '42')",
  },
  response: {
    status: "matching(equalTo, 'CHARGE_STATUS_APPROVED')",
    amount_cents: "matching(integer, 2599)",
  },
};
 
await pact
  .addSynchronousInteraction("a successful charge")
  .given("customer 7 has a valid card")
  .usingPlugin({ plugin: "protobuf" })
  .withPluginContents(JSON.stringify(contents), "application/grpc")
  .startTransport("grpc", "127.0.0.1")
  .executeTest(async (transport) => {
    const client = new PaymentsClient(`127.0.0.1:${transport.port}`);
    const result = await chargeOrder(client, "42");
    expect(result.totalCents).toBe(2599);
  });

Three details make this contract useful:

• It includes only the fields Orders actually uses. Orders does not read charge_id, so the contract does not mention it. Payments remains free to change it.
• It uses matchers, not exact values. matching(integer, 2599) means “an integer, for example 2599”. A contract that pins exact values breaks for reasons that have nothing to do with compatibility.
• It names a provider state. given("customer 7 has a valid card") tells Payments what data to set up before replaying the interaction. The provider implements a small handler for each state.

This contract contains amount_cents. When Payments v9 removes that field and runs verification, the interaction fails because the response has no field 3. The failure from the opening section is now caught in the Payments pipeline, before deployment.

What does not belong in a contract

Contract tests check the shape and meaning of the interaction. They do not check the provider’s business logic. “A declined card returns CHARGE_STATUS_DECLINED” is a reasonable contract interaction, because Orders branches on it. “A charge above the fraud threshold triggers a manual review” is Payments’ own behavior and belongs in Payments’ own tests. When contracts start encoding provider logic, every internal change to the provider breaks consumer contracts, and teams stop trusting them.

Verifying on the provider side

Payments verifies contracts fetched from the broker. Four settings do most of the work. In Pact JS they look like this:

const options = {
  provider: "payments",
  providerVersion: process.env.GIT_SHA,
  providerVersionBranch: process.env.GIT_BRANCH,
  consumerVersionSelectors: [
    { mainBranch: true },          // what consumers are about to ship
    { deployedOrReleased: true },  // what consumers are running now
  ],
  enablePending: true,
  includeWipPactsSince: "2026-08-01",
  publishVerificationResult: process.env.CI === "true",
};

• Consumer version selectors decide which contracts to verify. deployedOrReleased is the important one. It verifies the contracts of the consumer versions that are actually deployed in any environment, not only the latest code on main.
• Pending pacts stop a new consumer expectation from breaking the provider’s build. The build fails only when a change breaks a contract the provider previously supported.⁵
• Work-in-progress pacts automatically include newly changed contracts in verification, so consumers get feedback without the provider team editing its configuration.⁶
• Publishing only from CI keeps verification results trustworthy, because a developer’s local run never marks a version as compatible.

The deployment gate: can I deploy?

Verification results alone do not prevent the opening failure. The Payments pipeline could verify against Orders’ main branch, pass, and still deploy into an environment running an older Orders version. The broker closes this gap by also recording what is deployed where.

Each pipeline records deployments after they succeed:

pact-broker record-deployment \
  --pacticipant payments --version "$GIT_SHA" --environment production

Before deploying, each pipeline asks the broker whether the version it is about to deploy has a successful verification with every integrated application version currently in that environment:⁷

pact-broker can-i-deploy \
  --pacticipant payments --version "$GIT_SHA" --to-environment production

The broker answers from its matrix, the table of every consumer version and provider version that have been verified against each other.

Reading the matrix for the example:

• Production runs Orders v42 and Payments v8.
• Can Payments v9 deploy? The pair Orders v42 plus Payments v9 failed verification, because v42 still reads amount_cents. The answer is no.
• Can Orders v43 deploy? Orders v43 reads the new amount field, and Payments v8 sends both fields. The pair Orders v43 plus Payments v8 passed. The answer is yes.
• After Orders v43 is deployed, the production pair becomes Orders v43 plus Payments v8. Asking again whether Payments v9 can deploy checks Orders v43 plus Payments v9, which passed. The answer is now yes.

The gate did more than block a bad deploy. It worked out the safe deployment order without any cross-team coordination meeting.

Where the gate runs

Run can-i-deploy in two places:

1. Before every deployment, for every environment. This is the gate that matters. It checks against what is actually running there.
2. In the consumer’s pull request, against the provider’s main branch. This gives earlier feedback. A consumer PR that adds a new expectation fails until the provider supports it. Pact’s documentation describes this setup, including the webhook that triggers provider verification when a consumer publishes a changed contract.⁸

The full stage-by-stage gate policy for the series is in the hub’s section When tests run and what blocks delivery.

Version compatibility beyond one pair

Two versions run at the same time

Deployments are not instant. During a rolling deployment of Payments, some requests reach v8 and some reach v9. If the deployment fails halfway and is rolled back, v8 serves traffic again after v9 already has. A change is only safe if it works with the current version and the previous version on both sides, often called N/N-1 compatibility. Google’s protobuf guidance makes the same point: clients and servers are never updated at exactly the same moment, even when you try to update them together.⁹

The deployedOrReleased selector covers this for contracts, because both versions are recorded while they overlap. Some applications keep many versions alive for a long time, such as mobile apps and client SDKs. For those, use record-release instead of record-deployment. A release does not replace the previous version in the environment, so the broker keeps checking compatibility with every supported version.

Changing an interface safely

The opening failure came from a breaking change delivered in one step. The fix is the same parallel change pattern the series uses for database migrations: expand, migrate, contract.¹⁰ Applied to the amount field:

Three rules of thumb follow from this:

Consumers should also be tolerant readers: read only the fields they need and ignore everything else, so providers can add fields without coordination.¹¹

When consumer-driven contracts do not fit

Consumer-driven contracts depend on every consumer writing contract tests and on the provider being able to verify them. That breaks down for:

• Public or partner APIs, such as the Orders HTTP API that client applications call. You cannot ask external clients to publish contracts.
• Providers with many consumers, where verifying every consumer’s contract and maintaining provider states for all of them becomes expensive.
• Third-party APIs you depend on, because the third party will not verify your contracts.

For these boundaries, the provider owns a schema and every change to it is checked for breaking changes on the pull request.

Breaking-change detection for protobuf

buf breaking compares the current .proto files with a previous version, such as the main branch, and fails if a change is incompatible.¹²

# buf.yaml
version: v2
modules:
  - path: proto
breaking:
  use:
    - FILE

buf breaking --against '.git#branch=main'

Buf groups its rules into four categories, from strictest to most lenient: FILE, PACKAGE, WIRE_JSON and WIRE. FILE and PACKAGE protect generated source code, so consumers’ builds do not break. WIRE_JSON and WIRE protect only the encoded messages. Buf recommends WIRE_JSON as the minimum, because JSON encoding is common across transports, and uses FILE as its default.¹³ Pick the category that matches what consumers actually depend on. Payments shares generated stubs with Orders, so it should use FILE.

The categories answer different questions. A wire-level check confirms that encoded messages can still be decoded. It does not know whether a consumer still needs a field. Deleting amount_cents while reserving its number is wire-safe, and that is exactly the change that broke Orders. A schema check proves the shape is compatible. A contract test proves the consumer still gets what it uses. Where you can run both, run both.

Breaking-change detection for OpenAPI

For HTTP APIs, oasdiff compares two OpenAPI documents and reports breaking changes at two levels. ERR marks definite breaking changes, such as a removed endpoint. WARN marks potential ones that cannot be confirmed automatically.¹⁴

oasdiff breaking openapi-main.yaml openapi-pr.yaml --fail-on ERR

Run it on every pull request that touches the spec, against the version on main. If the spec is generated from code, generate it in CI and diff the result, so the check covers what the service really exposes rather than a hand-maintained file that may have drifted.

Costs and trade-offs

Contract testing is not free, and teams abandon it when the costs are hidden.

• Provider states are code to maintain. Every given(...) needs a handler that sets up data. Keep states few and generic, such as “customer has a valid card”, rather than one per test.
• It requires a broker and pipeline discipline. can-i-deploy is only as accurate as the deployments recorded. A single deploy that skips record-deployment makes the matrix wrong for that environment.
• Adoption has to be two-sided. A contract that no provider verifies gives false confidence. Start with one consumer-provider pair that has caused real incidents, and prove the value there before expanding.
• Contracts do not replace integration tests. They check the interaction, not the infrastructure behind it: timeouts, TLS configuration, authentication and real network behavior still need integration tests.
• Over-specified contracts become brittle. Exact values, fields the consumer does not read and provider business rules all make contracts fail without a real incompatibility.

In return, you remove the need for a shared environment where every service must be deployed together before anyone can release. That coordination bottleneck is the problem can-i-deploy was designed to remove.⁷

Tooling

Checklist

• Consumer contracts include only the fields and interactions the consumer uses.
• Contracts use type matchers, not exact values.
• Providers verify contracts from mainBranch and deployedOrReleased consumer versions.
• Pending and work-in-progress pacts are enabled on the provider’s main branch.
• Verification results are published only from CI.
• Every deployment runs can-i-deploy for its target environment and records the deployment afterward.
• Deleted protobuf field numbers and names are reserved.
• buf breaking or oasdiff runs on every pull request that changes a schema.
• Breaking interface changes go through expand, migrate and contract.

Related articles

• From Code to Production, the hub for this series.
• Testing Async Messaging and Schema Evolution: the same compatibility problem for events, with schema registries.
• Database Migrations, Rollback, and Roll-Forward: parallel change applied to databases.
• Testing a Single Service: Mocks, Fakes, and Real Dependencies: why mocks drift, and when they are still the right choice.

References