When to Use
Invoke this before upgrading a broker SDK version or pulling a new API specification (Binance, Coinbase, IBKR Client Portal and similar publish OpenAPI documents). Broker releases introduce silent breaking changes — a removed nested response field, a parameter that quietly became mandatory, a new order status the state machine has never seen. This skill diffs two schema snapshots and classifies what changed, so a CI job can fail the build before the change reaches an order path.
The tool is a gate, and its failure modes are asymmetric: a false positive costs a developer a few minutes, a false negative ships a broken integration. Everything about its classification is biased accordingly.
When NOT to Use
- As proof a release is safe. It compares structure only. Rate limits, auth scope changes, altered matching-engine behaviour, changed rounding, new error codes returned in a 200 body — none are expressible in a schema, and none will appear in the report. A clean diff means "nothing structural broke", not "safe to deploy".
- On specifications it cannot fully resolve. Only local (
#/...) references are followed. External and remote$refs are reported asUNRESOLVED_REF— that region was not compared, and treating the report as complete when one is present is a mistake. - As a Swagger 2.0 differ.
#/definitions/...references resolve, but Swagger 2.0's body parameters and top-levelconsumes/producesare not modeled; the request-body logic assumes OpenAPI 3.xrequestBody.content. Convert 2.0 documents to 3.x first. - As a file loader. It takes parsed Python dictionaries. Reading and parsing JSON or YAML is the caller's job.
- For composition keywords.
oneOf,anyOf,allOfanddiscriminatorare not evaluated; schemas using them will diff only at the level the tool can see.
Prerequisites
- Baseline (older) and target (newer) API schemas, parsed into dictionaries.
- Both documents complete, including the
components/definitionssections the$refs point at — a spec split across files must be bundled first, or references will come back unresolved.
Workflow
-
Load both documents and let the differ reject unusable input.
diff_schemasraisesSchemaDiffErrorwhen a document is not a mapping or has nopaths. This is deliberate: a failed download or a wrong path yields an empty document, and a differ that shrugs and reports zero changes turns the gate green at exactly the moment it matters. -
Diff endpoints. Removed paths and removed methods are
CRITICAL_BREAKING. Only real HTTP methods are treated as operations — a Path Item Object also legally carriesparameters,servers,summary,descriptionand$ref, and path-levelparametersare diffed as shared across every operation. -
Resolve
$refbefore comparing anything. Real broker specs describe payloads almost entirely through references, and a$refschema carries notype,propertiesorenumof its own. Resolution follows#/components/schemas/...and#/definitions/...against the document each side came from, with cycle protection for self-referential models. -
Treat absence as a change. A removed response status code, a removed request or response content type, and a removed
requestBodyare all breaking and all invisible to a differ that walks only the keys present on both sides. -
Check requirement transitions in both directions. A request field or parameter moving into
requiredbreaks callers that omit it. A response field moving out ofrequiredbreaks parsers that assume it is present. Both matter; they are not the same check. -
Classify enums by direction. A request enum constrains what the client may send, so removing a value is breaking. A response enum constrains what the client must handle, so adding a value is breaking — a new order status silently breaks an exhaustive state machine. Newly imposing a request constraint, and dropping a response constraint, are breaking too.
-
Gate the build.
report.exit_codeis 0 when compatible and 1 otherwise;report.format_report()renders the findings severity-first.is_compatibleis False if any change isMEDIUM_BREAKINGor higher.
Full procedure: see
references/workflows.md. Severity matrix and classification rationale: seereferences/standards.md. Printable pre-flight checklist: seeassets/checklist.md.
Common Pitfalls
- Comparing
$refschemas without resolving them. Both sides look like empty objects, every check is skipped, and a release that deleted an entire response model reports clean. This is the single most likely way to get a false green. - Ignoring an
UNRESOLVED_REFfinding. It is informational in severity but it means a region of the schema was never compared. Bundle the spec and re-run. - Only diffing keys present on both sides. Removals are the breaking changes; a
loop written as
if key in new: compare(...)cannot see any of them. - Treating enum changes as direction-agnostic. Flagging every set difference raises false alarms on request widenings while missing the response additions that actually break consumers.
- Assuming a scalar
type. OpenAPI 3.1 allowstype: ["object", "null"]where 3.0 usednullable: true. An equality test against the literal"object"silently skips property diffing, and comparing the two spellings reports a mutation that never happened. - Treating every key under a path as an HTTP method.
parametersis a list andsummaryis a string; calling.get()on them raises on a perfectly valid document. - Letting an empty or malformed document produce a clean report.
- Unbounded recursion on self-referential models.
Order.parent → Orderis ordinary, and resolving references without a cycle guard hangs the build. - Reading a clean report as deployment approval. Structure is not behaviour.
Verification
- Run the unit suite and confirm every test passes:
python -m unittest discover -s skills/broker-api-changelog-diffing-tool/scripts - Build a fixture whose response model sits behind a
$ref, delete a field from the referenced component, and confirmREMOVED_RESPONSE_FIELDis reported. A differ that passes every inline-schema test can still fail this one, which is the case that matters. - Confirm two empty documents raise
SchemaDiffErrorrather than reporting compatible. - Confirm a self-referential model terminates.
- Confirm direction-aware enum behaviour: adding a response enum value is breaking; adding a request enum value is not.
- Confirm a Path Item Object carrying
parametersandsummarydoes not raise. - Mutate fixtures by deep-copying a baseline and changing exactly one thing, so a test for one change cannot accidentally introduce another.