Skip to content

Dependency Vulnerability Scanning In CI

dependency-vulnerability-scanning-in-cisource

Use as the CI policy gate behind a real scanner such as pip-audit or OSV-Scanner: matches a dependency inventory against advisories using PEP 440 ordering and fails the build on critical or high findings.

Domain
Deployment & Ops
Version
1.1.0
Reading
5 min
Hands off to
4
Handed off from
2
License
Apache-2.0
Coverspip-auditOSV DatabasePyPA Advisory DBCycloneDXPython Dataclasses

When to Use

Use this skill in continuous integration (CI/CD) pipelines, automated build checks, and production trading system deployments. Third-party Python dependencies (such as requests, urllib3, cryptography, pyyaml) frequently contain security vulnerabilities (Remote Code Execution, Denial of Service, SQL injection). Deploying vulnerable packages to production trading servers exposes trade execution APIs and private keys to compromise. This module evaluates a dependency inventory against advisory records and hard-fails CI builds on CRITICAL or HIGH severity flaws.

When NOT to Use

  • As a replacement for pip-audit or OSV-Scanner. This engine does not fetch advisories or resolve a dependency graph. It is the policy gate you put behind a real scanner: run pip-audit/OSV-Scanner, feed its findings in via register_advisory(), and let this engine apply consistent severity and fail-closed rules. Using it alone means scanning against whatever advisories you happened to register.
  • For transitive dependency discovery. It scans the inventory you hand it. If you pass only top-level requirements.txt entries, nested transitive packages are never examined — export a fully resolved inventory (pip freeze, a lockfile, or the scanner's own resolved list) instead.
  • For non-Python ecosystems. Version ordering is PEP 440 and purls are emitted as pkg:pypi/.... npm/Maven/Go versions follow different ordering rules and will be mis-compared.
  • As an SBOM system of record. generate_sbom() emits a valid CycloneDX 1.6 document for the inventory it was given; it does not capture licences, hashes, provenance, or build attestations that a full SBOM toolchain produces.

Prerequisites

  • A fully resolved dependency inventory (package_name, current_version) — not just direct dependencies.
  • CVE advisory records (cve_id, package_name, vulnerable_below_version, cvss_score, severity, fixed_in_version, and, where the advisory states one, introduced_in_version).

Workflow

  1. Lockfile & Inventory Parsing:

    • Parse package names and installed versions from a resolved lockfile.
    • An empty inventory raises ValueError rather than reporting a clean scan — a parser that silently returned nothing must not read as "no vulnerabilities".

  2. Advisory Registration:

    • Load advisories from the OSV/PyPA feed. An empty advisory database also raises ValueError: scanning against zero advisories always reports zero findings, which is the "stale CVE database" failure mode wearing a green badge.
    • Package names are normalised per PyPI rules (lowercased, _ → -) before matching, so an advisory for ruamel-yaml still matches an installed Ruamel_YAML.

  3. Affected-Range Evaluation:

    • Compare the installed version against the advisory range using PEP 440 ordering, not string or naive integer comparison. 2.31.0rc1 sorts before 2.31.0 and is therefore still vulnerable; 2.31 and 2.31.0 are equal.
    • Honour introduced_in_version where the advisory has a lower bound — CVE-2023-32681 affects requests >= 2.3.0, < 2.31.0, so a project pinned to 2.2.0 is not affected and must not be flagged.
    • If a version string cannot be parsed as PEP 440, fail closed: report the package as affected and record a scan warning. A gate must never clear a version it could not understand.

  4. Severity Resolution (fail-closed):

    • If the advisory's severity label is canonical (CRITICAL/HIGH/MEDIUM/LOW/NONE), use it.
    • If it is not — GitHub Security Advisories say MODERATE, feeds carry stray whitespace, some records have none at all — derive severity from the CVSS base score using the FIRST CVSS v3.1 rating scale. Do not bucket an unrecognised label into LOW.
    • If label and score disagree, gate on the more severe of the two and emit a warning.
    • If neither resolves, classify UNKNOWN, which blocks the build.

  5. CI Pipeline Gate & Remediation:

    • If CRITICAL, HIGH, or UNKNOWN findings exist $\implies$ set is_ci_build_passed = False and block deployment.
    • Emit exact package upgrade remediation directives (e.g. "Upgrade requests 2.25.0 $\to$ 2.31.0").

  6. Audit Report & SBOM Generation:

    • Output a structured VulnerabilityScanReport (including scan_warnings, which must be surfaced in the CI log, not swallowed).
    • Call generate_sbom() for a CycloneDX 1.6 JSON document. Output is deterministic — the serialNumber is a UUIDv5 over the component set and the timestamp is opt-in — so the SBOM diffs cleanly between builds.

Full procedure: see references/workflows.md. Standards reference: see references/standards.md. Printable pre-flight checklist: see assets/checklist.md.

Common Pitfalls

  • Ignoring Transitive Sub-Dependencies: Scanning top-level packages while missing vulnerable transitive dependencies nested in requirements.txt.
  • Soft Warnings on Critical RCE Vulnerabilities: Issuing non-blocking warnings for CRITICAL (CVSS $\ge 9.0$) vulnerabilities, allowing compromised builds to reach production.
  • Outdated CVE Databases: Running CI security scans without updating the advisory database feed. A scan against a stale or empty feed returns "0 vulnerabilities" — indistinguishable from a genuinely clean build unless you assert the advisory count.
  • Comparing Versions as Strings or Split Integers: "2.9.0" < "2.31.0" is False lexicographically, and int()-splitting explodes on any PEP 440 pre/post/dev suffix. Both silently clear a vulnerable package. Use PEP 440 ordering, and treat an unparseable version as affected rather than clean.
  • Bucketing Unknown Severity Labels as Low: An if/elif chain over CRITICAL/HIGH/MEDIUM with an else: low fallback turns every non-canonical label — MODERATE, "CRITICAL " with a trailing space, an empty string — into a LOW finding that passes the gate, no matter how high the CVSS score is.
  • Treating "Vulnerable Below X" as the Whole Advisory: Advisories are ranges. Modelling only the upper bound flags every older release, including ones predating the vulnerable code, and buries real findings in false positives.

Verification

  • Instantiate DependencyVulnerabilityScannerEngine. Register CVE-2023-32681 (requests >= 2.3.0, < 2.31.0; NVD CVSS v3.1 base score 6.1 MEDIUM). Audit requests==2.25.0 with numpy==1.26.4: the scanner flags one MEDIUM finding and recommends upgrading to 2.31.0, and — because MEDIUM is below the blocking threshold — is_ci_build_passed stays True. Audit requests==2.2.0 and confirm it is not flagged (below the advisory's introduced bound).
  • Register an advisory with severity "MODERATE" and CVSS 9.8; confirm it resolves to CRITICAL and blocks the build.
  • Confirm scan_dependencies([]) and a scan with no registered advisories both raise ValueError.
  • Confirm generate_sbom() returns bomFormat="CycloneDX", specVersion="1.6", an RFC 4122 urn:uuid: serial number, and one library component per package.
  • Run python -m unittest discover -s skills/dependency-vulnerability-scanning-in-ci/scripts.

Verify it, from the repository root

python -m unittest discover -s skills/dependency-vulnerability-scanning-in-ci/scripts

Hands off to 4

Skills this document names, usually in When NOT to Use, as the owner of a case it excludes.

Supervision, promotion, chaos

Auth, tokens, idempotent orders

Keys, wallets, withdrawals

Handed off from 2

Skills that name this one as the place a case belongs. The reverse edges of the graph.

Supervision, promotion, chaos

SEC, FINRA, MiFID II, FCA, SEBI