schema-impact
schema-impact
Section titled “schema-impact”A schema diff says a field was renamed. That answers the wrong
question: it does not say which files still name the old field.
This command makes the join through the generated-code boundary.
A .proto file changes between two refs; the generated Go or
Python code in the tree maps each schema element to a generated
symbol; the files importing that generated code and naming the
symbol are the consumers. Every consumer carries its evidence
tier, and the tier decides what the report can gate on.
The setup below is a small repo you can copy to follow along: a
proto file, its generated Python modules, and a consumer. Commit
and tag v1, then rename name to full_name, delete email,
and delete the RED enum value, and commit and tag that as v2:
git init -qmkdir -p apicat > api/user.proto <<'EOF'syntax = "proto3";package api;message User { string name = 1; int32 age = 2; string email = 3;}enum Color { COLOR_UNSPECIFIED = 0; RED = 1; BLUE = 2;}service UserService { rpc GetUser (GetUserRequest) returns (User);}message GetUserRequest { string user_id = 1;}EOFprintf '# Generated by the protocol buffer compiler. DO NOT EDIT!\n' > api/user_pb2.pyprintf '# Generated by the protocol buffer compiler. DO NOT EDIT!\n' > api/user_pb2_grpc.pycat > consumer.py <<'EOF'from api.user_pb2 import User, Color
def make(): return User(name="Ada", age=36, email="ada@example.com")
def favorite(): return Color.REDEOFgit add . && git commit -qm v1 && git tag v1cat > api/user.proto <<'EOF'syntax = "proto3";package api;message User { string full_name = 1; int32 age = 2;}enum Color { COLOR_UNSPECIFIED = 0; BLUE = 2;}service UserService { rpc GetUser (GetUserRequest) returns (User);}message GetUserRequest { string user_id = 1;}EOFgit add . && git commit -qm v2 && git tag v2sephera schema-impact --path . --from v1 --to v2sephera schema-impact --path . --from v1sephera schema-impact --path . --from v1 --to v2 --format jsonsephera schema-impact --path . --from v1 --to v2 --fail-on 1--from names the older side and is required. --to names the
newer side; when it is absent, the newer side is the working
tree — staged, unstaged, and untracked files included — which is
the form a branch review takes before it is pushed. When --to
is present, both sides are checked out into temporary worktrees,
so the working tree is left untouched either way.
The first command above, run against the demo repo, prints:
# Schema impact: `v1` → `v2`
## Breaking changes (3)
### `api.Color.RED` — removed
Schema file: `api/user.proto`
Proven consumers:
- `consumer.py` — via enum constant
### `api.User.email` — removed
Schema file: `api/user.proto`
Proven consumers:
- `consumer.py` — via constructor keyword
### `api.User.name` — renamed from `name` to `full_name`
Schema file: `api/user.proto`
Proven consumers:
- `consumer.py` — via constructor keywordThe JSON form reports the same breaks as data, with the same proven and name-match split:
{ "from_ref": "v1", "to_ref": "v2", "breaks": [ { "schema_file": "api/user.proto", "element": "api.Color.RED", "change": "removed", "note": null, "proven": [ { "file": "consumer.py", "via": "enum_const" } ], "named": [], "downstream": [] }, { "schema_file": "api/user.proto", "element": "api.User.email", "change": "removed", "note": null, "proven": [ { "file": "consumer.py", "via": "constructor_kwarg" } ], "named": [], "downstream": [] }, { "schema_file": "api/user.proto", "element": "api.User.name", "change": { "renamed": { "from": "name", "to": "full_name" } }, "note": null, "proven": [ { "file": "consumer.py", "via": "constructor_kwarg" } ], "named": [], "downstream": [] } ], "added": [], "coverage": []}--fail-on counts proven consumers only and exits 2 at the
threshold. The demo repo has three proven breaks, so
--fail-on 1 exits 2; a break with only name-match consumers
never trips it, and a break nobody uses does not trip it
either. The report is still printed; only the exit code
changes.
What counts as a break
Section titled “What counts as a break”Field numbers are the identity anchor. A field with the same
number and type under a new name is a rename — a proof, not a
guess — and its consumers are the files naming the old name. A
number reused by a different type is removal plus addition, never
a rename. Messages, enums, and services have no numbers, so their
renames read as removal plus addition, exactly like breaking’s
move rule: the tool does not guess identity from similar field
sets. The kinds:
- Removed — the element is gone from the new side.
- Renamed — field or enum value only: same number, same type (fields), old name absent.
- TypeChanged — same name and number, different type. A scalar swap and a message-type swap are both this kind.
- NumberChanged — same name, different number: a wire break and a source break, reported as its own kind, never folded into a rename.
- LabelChanged —
optionalversusrepeatedon the same name and number. - OneofChanged — the field’s oneof membership changed.
- InputChanged / OutputChanged — an rpc method’s input or output message changed.
- Added — informational only, in its own section. A new field cannot break existing consumers, so additions are never gated and never listed as broken.
A map field’s change is reported at the map field itself; the
synthetic entry message (Msg.FieldEntry) is never a
reportable element. A field inside a oneof is reported at the
field, not as a wrapper break.
How consumers are found, by tier
Section titled “How consumers are found, by tier”- Proven, message-level (A1) — the file imports the generated module and names the message, enum, or service symbol: a Python from-import, a Python qualified use, or a Go package-qualified selector.
- Proven, field-level (A2) — a Go struct literal, a Python constructor keyword, an enum constant, or a method call names the element in a position tied to the generated type. A method call is proven evidence for a method break; a file that merely constructs a client is not, because constructing a client does not call the method.
- Flagged, name-match (A3) — the file imports the
generated module and names the field, but the receiver is
unknown: a field access on an unknown receiver, or a string
literal equal to the field’s
json_name. A3 is labelled weaker evidence in the report and is never gated.
Tier C — a bare substring match with no import gate and no syntactic anchor — is refused outright. There is no opt-in that enables it: a weak tier users can enable becomes the tier everyone enables and then distrusts.
Consumers are old code, so the join reads old names: a rename’s consumers name the old field and the old JSON key, and a removed element’s consumers name the element the new side no longer has. The consumer catalog is therefore built from the old schema, while generated-file discovery runs on the new tree’s texts. Downstream files — files importing a proven consumer — are listed apart, never as consumers themselves.
Scope, formats, and limits
Section titled “Scope, formats, and limits”--path scopes the comparison the way it does for breaking:
with no --to, both sides are restricted to --path;
with --to, both sides are whole-tree checkouts. The command
needs a git repository: a tree that is not one is an error, not
an empty report. --url is refused, because a remote checkout
is one ref already and has no second ref to compare against.
There is no SARIF output in v1.
Proto3 only. A proto2 file is an error naming the file, never a partial report — and a file without a syntax declaration defaults to proto2, so it errors too. Generated code must be in the tree for the proven tiers: a repo that generates at build time gets a coverage note, never a “0 consumers” claim. The same rule covers a break whose element maps to no generated module in the new tree: the report says the join could not see its consumers, per element.
One mapping limit is stated rather than hidden. Go modules
map by the struct and interface names their generated file
declares, so a Go message removed and regenerated — the
struct declaration gone with it — has no module to join
through, and its break is listed with that note. Python
modules map by path (api/user.proto to api/user_pb2.py),
so a Python removal joins even after regeneration. Both
mappings handle the standard protoc layout, where generated
code lives under its own root: a Python file under
python/ maps to the proto under proto/ with the same
path relative to each root, and a Go file under go/ is
disambiguated from a same-named struct in another package
by the trailing path segments the two directories share.
A json_name-only change is not a break in v1: the wire
name moved but the source contract did not, and v1 has no
kind for it.
A second limit follows from the proto3-only rule: one
proto2 file anywhere fails a whole-tree run, because the
strict parser errors rather than silently skipping a file
whose elements it cannot diff. Repos that vendor a proto2
file — the usual case is a protovalidate predefined-rules
file — can scope the run to a proto3-only subtree with
--path in the no---to form, or exclude the file with
--ignore.
Why this is not buf breaking
Section titled “Why this is not buf breaking”buf breaking compares two versions of a Protobuf schema and
reports incompatibilities at proto locations — schema against
schema. It does not name the in-repo files that still use the
changed element. SpecShield’s “consumer-aware” gate is a
registered-consumer deploy gate: it blocks a merge when a
change would break a consumer service registered in its
registry, which is contract testing against declared
dependents, not a static join from a schema change to the
code files consuming it. GraphQL Inspector’s “coverage”
counts how often schema types and fields appear in supplied
GraphQL documents and annotates the schema with those stats;
it does not map a schema change to application-code files.
oasdiff is a spec-to-spec differ with, in a vendor
comparison’s wording, “no concept of consumer impact
analysis”. Nothing found joins a schema change to exactly the
in-repo code that consumes it, across languages, with
evidence tiers — that join is this command.
a field is renamed
old code still calls the old name
the break finds its files