Schema Evolution Without Breaking Consumers: Avro, Protobuf, and Compatibility Modes

You will learn how Avro and Protobuf resolve old and new schemas, which changes are safe under BACKWARD, FORWARD, and FULL compatibility, and how to evolve a production schema — with a registry, expand-and-contract, and CI contract checks — without breaking consumers.

Why schema evolution breaks consumers

A schema is a contract between code that writes data and code that reads it. Producers and consumers rarely deploy at the same moment, and stored data outlives any deployment. At any time you can have new producers writing to old consumers, old producers writing to new consumers, and new consumers reading old data.

A change that looks harmless in review — renaming a field, changing a type, removing a value — becomes breaking when it meets a different schema version on the other side. Compatibility modes name which version pairing you promise to support, and a registry enforces that promise before broken data is produced.

The two formats solve this differently. Avro resolves a writer schema against a reader schema at read time. Protobuf relies on field numbers that stay stable forever, so old and new code can parse the same bytes and ignore what they do not understand.

How Avro resolves reader and writer schemas

Avro data is written with a writer schema and read with a reader schema. When they differ, the reader resolves the writer schema field by field:

• If a field exists in both, its value is read, with type promotion if needed.
• If a field exists only in the writer schema, its value is skipped.
• If a field exists only in the reader schema, the reader must have a default for it. Without one, resolution fails.
• If a field was renamed, aliases on the reader field can map it to the old writer name.

Adding a field is therefore safe for new readers only if it has a default, because new readers encounter old data without it. Removing a field is safe for old readers only if it had a default in the old schema, because old readers encounter new data without it.

{
  "type": "record",
  "name": "User",
  "namespace": "com.example.events",
  "fields": [
    { "name": "user_id", "type": "string" },
    { "name": "email", "type": "string" },
    { "name": "display_name", "type": ["null", "string"], "default": null }
  ]
}

{
  "type": "record",
  "name": "User",
  "namespace": "com.example.events",
  "fields": [
    { "name": "user_id", "type": "string" },
    { "name": "email", "type": "string" },
    { "name": "display_name", "type": ["null", "string"], "default": null },
    { "name": "country_code", "type": ["null", "string"], "default": null }
  ]
}

Version 2 adds country_code with a null default. A version 2 consumer reads version 1 data and sees null. A version 1 consumer reads version 2 data and skips the field. The change is both backward and forward compatible.

Type promotion is deliberately narrow. Avro permits reading int as long, float, or double; long as float or double; and float as double. It also permits string to bytes and bytes to string when the bytes are valid in that encoding. Reverse directions are not promotions: reading a long as an int can truncate, so resolution rejects it. Changing string to int is not evolution — it needs a new field, subject, or topic.

Renaming requires an alias:

{
  "type": "record",
  "name": "User",
  "namespace": "com.example.events",
  "fields": [
    { "name": "user_id", "type": "string" },
    { "name": "email_address", "type": "string", "aliases": ["email"] },
    { "name": "display_name", "type": ["null", "string"], "default": null },
    { "name": "country_code", "type": ["null", "string"], "default": null }
  ]
}

Logical types add semantics on a primitive type: timestamp-millis and timestamp-micros on long, decimal on bytes or fixed, uuid on string, date on int. Treat it as part of the contract. Changing timestamp-millis to timestamp-micros keeps the underlying long, so data parses but values are interpreted at a different scale.

How Protobuf evolves without breaking wire compatibility

Protobuf does not resolve two schemas. Each field is encoded with a field number, and that number is its identity on the wire. Names matter for source code, JSON mapping, and text format, but binary encoding carries only numbers and wire types.

One discipline outweighs everything else: once a field number is assigned, never reuse it for a different meaning. When you remove a field, reserve its number and name so neither can be recycled.

syntax = "proto3";
 
package com.example.events;
 
message User {
  string user_id = 1;
  string email = 2;
  string display_name = 3;
}

syntax = "proto3";
 
package com.example.events;
 
message User {
  string user_id = 1;
  string email_address = 2;
  string display_name = 3;
  string country_code = 4;
 
  reserved 5, 6;
  reserved "legacy_status", "temp_flag";
}

Field 2 was renamed from email to email_address. Binary data is unaffected because number and type are unchanged. The rename is still hazardous: JSON uses the field name by default, generated accessors change and break calling code, and stored mappings or text-format fixtures that use the old name stop working. Prefer adding a new field over renaming one whose name is exposed outside the binary contract.

Other rules follow from the wire format:

• Adding a field with a fresh number is safe. Old readers ignore it when their runtime preserves unknown fields; new readers see a zero value, empty string, or empty list in old data.
• Changing a type is safe only within wire-compatible groups and only when the new interpretation matches old values. int32 to int64 is wire-compatible; string to bytes keeps the wire type but changes validation and meaning; moving a field into or out of a oneof, or changing singular to repeated, can break semantics even when bytes decode.
• Renumbering a field is always breaking, even if name and type stay the same.
• Enum values have numbers too. Add values with new numbers, never renumber, and reserve removed numbers and names.

Unknown-field behaviour varies by language, runtime, and format. Modern runtimes usually preserve unknown fields when reserializing, letting an intermediary pass them through. JSON parsing may instead reject them depending on options. Test pass-through; do not assume it.

Compatibility modes: what each one actually guarantees

A mode is a promise about which pairings must work. It says nothing about whether a change is semantically sensible — only whether readers and writers on different versions can exchange data.

The transitive distinction is most often missed. A non-transitive check compares a candidate only with the immediately preceding version. Individually compatible steps can still break a reader jumping two versions: remove a field, then add a different field with the same name and a different type. Each step passes against its neighbour, while the first and third versions are incompatible. If consumers lag, you replay old data, or producers roll back, you need the transitive form.

Choose from deployment reality. BACKWARD suits consumer-first streaming. If producers may roll back while consumers stay upgraded, you also need FORWARD. If upgrade order is uncontrolled, require FULL_TRANSITIVE.

Enforcing compatibility with a schema registry

A registry stores every version under a named subject, assigns it an identifier, and rejects a new version that violates the subject’s compatibility mode. Producers and consumers fetch schemas by identifier instead of embedding copies in every message.

Subject naming decides what the contract covers: a topic-based strategy ties a subject to one topic, a record-based strategy ties it to a fully qualified record name, and a combined strategy ties it to both. Pick before the first registration — changing later splits one contract across two subjects.

Conceptually, the registry offers a small REST surface: register a version under a subject, fetch a schema by identifier or by subject and version, list subjects and versions, and read or update compatibility configuration per subject or globally. Pair it with access control, or registration is documentation, not enforcement.

The expand-and-contract pattern

Most breaking changes can be decomposed into compatible steps:

1. Expand. Add the new field, enum value, or message alongside the old one. Give new Avro fields a default. Keep the old field populated. Deploy consumers that read both forms and prefer the new one when present.
2. Migrate. Deploy producers writing the new form. Dual-write both forms from a single source of truth, or backfill stored data, and monitor which form consumers still use.
3. Contract. When no consumer reads the old field and no producer writes it, remove it. In Protobuf, reserve its number and name. Register the contracted schema as a new version.

Dual-write from two code paths invites drift when one write succeeds and the other fails. Write from one place, make it idempotent where you can, and compare both streams during transition. Dual-read with explicit precedence — new field if present, otherwise old — and measure which form was used, so you have evidence before contracting.

For a change that cannot be decomposed — changing a field’s meaning, splitting one event into two, changing a key — create a new subject or topic, run old and new in parallel, and retire the old contract only after traffic reaches zero.

Testing contracts in CI

Run registry checks on every schema change, before merge, so breaking proposals fail at review time.

#!/usr/bin/env bash
set -euo pipefail
 
REGISTRY_URL="${SCHEMA_REGISTRY_URL:?set SCHEMA_REGISTRY_URL}"
SUBJECT="${1:?usage: check-compatibility.sh <subject> <schema-file>}"
SCHEMA_FILE="${2:?usage: check-compatibility.sh <subject> <schema-file>}"
 
jq -n --rawfile schema "$SCHEMA_FILE" \
  '{schema: $schema, schemaType: "AVRO"}' > /tmp/payload.json
 
curl --fail -s \
  -X POST \
  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  --data @/tmp/payload.json \
  "$REGISTRY_URL/compatibility/subjects/${SUBJECT}/versions/latest" | jq .

For Protobuf, add a diff check against previous .proto files flagging unreserved deletions, renumbering, and unsafe type changes.

Beyond the registry gate, add consumer-driven contract checks: keep each critical consumer’s expectations — fields it reads, enum values it handles, defaults it relies on — as tests in the producer repository, so a breaking change fails with that consumer named. Also encode with the old schema and decode with the new one, and the reverse, for every pairing your mode promises.

Failure modes that pass checks but break production

Default-value traps. An Avro reader invents the default for a missing field. If that default looks plausible — an empty string where absence meant something else, or a numeric default that is also a real measurement — downstream logic cannot distinguish invented from real data. Prefer a nullable union with a null default when absence carries meaning.

Enum additions breaking strict readers. Adding an enum value is structurally compatible because old data never contains it. New data does. A consumer with an exhaustive switch and no default branch, a strict JSON parser, or generated code that throws on an unrecognised value fails only when the new value first appears. Deploy tolerant consumers first, then produce the value.

String-to-enum coercion. Moving a free-form string to an enum looks like tightening, not breaking. Stored values that are misspelled, differently cased, or outside the new set have no valid mapping, and silently mapping them to a fallback corrupts reporting. Validate stored values first and handle unmapped values explicitly.

Silent data loss from dropped fields. A consumer that reads, transforms, and reserializes a message drops fields its schema does not know if its runtime does not preserve unknown fields. No error is raised; the field is gone downstream. Test intermediaries with unknown fields present.

Repurposed fields. Reusing a field for a new meaning because its type fits passes every check and corrupts historical comparison. Add a new field instead.

Verdict: which format and mode to choose

Choose Protobuf when you need strong code generation across many languages, compact encoding with stable field numbers, and one contract shared between streaming and request-response services. Choose Avro when data is record-oriented and long-lived — files, lakes, and replayable logs — and you want writer and reader schemas resolved explicitly with defaults and aliases. Either works with a registry; decide by how data is stored and read, not by syntax preference.

For mode, start with BACKWARD_TRANSITIVE for consumer-first streaming, and use FULL_TRANSITIVE when upgrade order is uncontrolled, rollback is expected, or old data is replayed. Use NONE only on subjects nobody else depends on.

Hold the same rules either way: never reuse a Protobuf field number, never add an Avro field without a default, never rename without an alias or a reservation, and never treat a passed check as proof a change is safe. Decompose changes with expand-and-contract, prove each pairing with encoded payloads in CI, and let the registry reject the rest.