kefeimo / volttron-core

Other
0 stars 0 forks source link

SBOM pilot task #3

Open kefeimo opened 5 months ago

kefeimo commented 5 months ago

SBOM basics write up

Strategy and Steps to Create a SBOM Pilot Demo for the "volttron-core" Repository

Significance of Using SBOM

A Software Bill of Materials (SBOM) is a comprehensive inventory of all components, libraries, and modules that are included in a piece of software. SBOMs are significant because they:

  1. Enhance Security: Identify and mitigate vulnerabilities by knowing exactly what components are in use.
  2. Ensure Compliance: Maintain compliance with open source licenses and regulatory requirements.
  3. Improve Maintenance: Facilitate easier updates and maintenance by understanding dependencies.
  4. Support Incident Response: Quickly assess and respond to security incidents by knowing affected components.

Implementation Strategy

The strategy involves creating an SBOM for the "volttron-core" repository both locally and via GitHub Actions for continuous integration. The process includes:

  1. Local Setup: Develop and test the SBOM generation process on a local machine using WSL Ubuntu 22.
  2. GitHub Actions Integration: Automate the SBOM generation process using GitHub Actions to ensure continuous monitoring and generation on code changes.

Local Setup Steps

  1. Install WSL and Ubuntu 22:
    • Ensure WSL is installed and set up with Ubuntu 22.
    • Update packages: sudo apt update && sudo apt upgrade
  2. Install Node.js and npm:

    curl -fsSL <https://deb.nodesource.com/setup_20.x> | sudo -E bash -
    sudo apt-get install -y nodejs
    
    • Generated files: None
  3. Install Python and pip:

    sudo apt-get install -y python3 python3-pip
    
    • Generated files: None
  4. Install cdxgen and depscan:

    sudo npm install -g @cyclonedx/cdxgen
    sudo pip install owasp-depscan
    
    • Generated files: None
  5. Clone the volttron-core repository:

    git clone <https://github.com/kefeimo/volttron-core.git>
    cd volttron-core
    
    • Generated files: None
  6. Generate SBOM and Vulnerability Data Report:

    cdxgen -t python .
    depscan --bom bom.json
    mv ./bom.vdr.json ./reports/bom.vdr.json || echo "No VDR was generated."
    
    • Generated files:
      • bom.json: SBOM file generated by cdxgen.
      • bom.vdr.json: Vulnerability Data Report (VDR) file generated by depscan.

GitHub Actions Integration

  1. Create Workflow File: Add a new GitHub Actions workflow in .github/workflows/sbom.yml.
  2. GitHub Actions Workflow Configuration:
# .github/workflows/sbom.yml
name: Generate SBOM and VDR

on:
  push:
    branches:
      - main
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  generate-sbom:
    runs-on: ubuntu-latest
    steps:
      - name: Check out repository
        uses: actions/checkout@v3

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'

      - name: Install cdxgen
        run: npm install -g @cyclonedx/cdxgen

      - name: Install depscan
        run: pip install owasp-depscan

      - name: Generate SBOM
        run: cdxgen -t python .
        # Generated files: bom.json

      - name: Generate Vulnerability Data Report
        run: |
          depscan --bom bom.json
          TIMEOUT=10
          while [ $TIMEOUT -gt 0 ]; do
            if [ -f "./bom.vdr.json" ]; then
              mv ./bom.vdr.json ./reports/bom.vdr.json
              echo "VDR file generated and moved to reports directory."
              break
            fi
            echo "Waiting for bom.vdr.json..."
            sleep 1
            TIMEOUT=$((TIMEOUT-1))
          done
          if [ $TIMEOUT -eq 0 ]; then
            echo "Timeout reached. No VDR file generated."
          fi
        # Generated files: bom.vdr.json (if VDR is generated)

      - name: Upload SBOM Artifact
        uses: actions/upload-artifact@v4
        with:
          name: sbom
          path: ./bom.json

      - name: Upload VDR Artifact
        uses: actions/upload-artifact@v4
        with:
          name: vdr
          path: ./reports

Running the Demo Locally

  1. Navigate to the Project Directory:

    cd path/to/volttron-core
    
  2. Run SBOM and VDR Generation Commands:

    cdxgen -t python .  # generate bom.json
    depscan --bom bom.json  # generate bom.vdr.json
    mv ./bom.vdr.json ./reports/bom.vdr.json || echo "No VDR was generated."
    
  3. Verify Outputs: Check for the bom.json and bom.vdr.json files in the project directory.

  4. Example output log

    $ depscan --bom bom.json  # generate bom.vdr.json
    
    ██████╗ ███████╗██████╗ ███████╗ ██████╗ █████╗ ███╗   ██╗
    ██╔══██╗██╔════╝██╔══██╗██╔════╝██╔════╝██╔══██╗████╗  ██║
    ██║  ██║█████╗  ██████╔╝███████╗██║     ███████║██╔██╗ ██║
    ██║  ██║██╔══╝  ██╔═══╝ ╚════██║██║     ██╔══██║██║╚██╗██║
    ██████╔╝███████╗██║     ███████║╚██████╗██║  ██║██║ ╚████║
    ╚═════╝ ╚══════╝╚═╝     ╚══════╝ ╚═════╝╚═╝  ╚═╝╚═╝  ╚═══╝
    
    INFO [2024-06-16 19:04:47,766] Performing regular scan for /home/kefei/project/volttron-core using plugin bom
    
                                                                Dependency Scan Results (BOM)                                                             
    ╔════════════════════════════════════════════════════════════════╤════════════════════════════════════╤════════════════════╤═══════════════╤═════════╗
    ║ Dependency Tree                                                │ Insights                           │ Fix Version        │ Severity      │   Score ║
    ╟────────────────────────────────────────────────────────────────┼────────────────────────────────────┼────────────────────┼───────────────┼─────────╢
    ║ cryptography@36.0.2 ⬅ CVE-2023-50782                           │ 🧾 Vendor Confirmed                │ 42.0.2             │ HIGH          │     7.5 ║
    ╟────────────────────────────────────────────────────────────────┼────────────────────────────────────┼────────────────────┼───────────────┼─────────╢
    ║ cryptography@36.0.2 ⬅ CVE-2023-49083                           │ 🧾 Vendor Confirmed                │ 41.0.6             │ MEDIUM        │     5.9 ║
    ╟────────────────────────────────────────────────────────────────┼────────────────────────────────────┼────────────────────┼───────────────┼─────────╢
    ║ pytest@6.2.5                                                   │ 📓 Indirect dependency             │                    │ HIGH          │     7.5 ║
    ║ └── py@1.11.0 ⬅ CVE-2022-42969                                 │                                    │                    │               │         ║
    ╟────────────────────────────────────────────────────────────────┼────────────────────────────────────┼────────────────────┼───────────────┼─────────╢
    ║ cryptography@36.0.2 ⬅ CVE-2023-0286                            │ 🧾 Vendor Confirmed                │ 39.0.1             │ HIGH          │     7.4 ║
    ╟────────────────────────────────────────────────────────────────┼────────────────────────────────────┼────────────────────┼───────────────┼─────────╢
    ║ cryptography@36.0.2 ⬅ GHSA-5cpq-8wj7-hf2v                      │                                    │ 41.0.0             │ LOW           │     2.0 ║
    ╟────────────────────────────────────────────────────────────────┼────────────────────────────────────┼────────────────────┼───────────────┼─────────╢
    ║ watchdog-gevent@0.1.1                                          │ 📓 Indirect dependency             │ 23.9.0             │ CRITICAL      │     9.8 ║
    ║ └── gevent@21.12.0 ⬅ CVE-2023-41419                            │                                    │                    │               │         ║
    ╟────────────────────────────────────────────────────────────────┼────────────────────────────────────┼────────────────────┼───────────────┼─────────╢
    ║ cryptography@36.0.2 ⬅ GHSA-jm77-qphf-c4w8                      │                                    │ 41.0.3             │ LOW           │     2.0 ║
    ╟────────────────────────────────────────────────────────────────┼────────────────────────────────────┼────────────────────┼───────────────┼─────────╢
    ║ cryptography@36.0.2 ⬅ CVE-2023-23931                           │                                    │ 39.0.1             │ MEDIUM        │     6.5 ║
    ╟────────────────────────────────────────────────────────────────┼────────────────────────────────────┼────────────────────┼───────────────┼─────────╢
    ║ cryptography@36.0.2 ⬅ GHSA-v8gr-m533-ghj9                      │                                    │ 41.0.4             │ LOW           │     2.0 ║
    ╟────────────────────────────────────────────────────────────────┼────────────────────────────────────┼────────────────────┼───────────────┼─────────╢
    ║ cryptography@36.0.2 ⬅ CVE-2024-0727                            │ 🧾 Vendor Confirmed                │ 42.0.2             │ MEDIUM        │     5.5 ║
    ╚════════════════════════════════════════════════════════════════╧════════════════════════════════════╧════════════════════╧═══════════════╧═════════╝

Integration with GitHub Actions

  1. Commit and Push Workflow:

    git add .github/workflows/sbom.yml
    git commit -m "Add SBOM generation workflow"
    git push origin main
    
  2. Verify GitHub Actions: Check the Actions tab in the GitHub repository to ensure the workflow runs on every push or pull request to the main branch.

  3. Example Action Log (i.e., of Generate SBOM and VDR)

    Run depscan --bom bom.json
      depscan --bom bom.json
      TIMEOUT=10
      while [ $TIMEOUT -gt 0 ]; do
        if [ -f "./bom.vdr.json" ]; then
          mv ./bom.vdr.json ./reports/bom.vdr.json
          echo "VDR file generated and moved to reports directory."
          break
        fi
        echo "Waiting for bom.vdr.json..."
        sleep 1
        TIMEOUT=$((TIMEOUT-1))
      done
      if [ $TIMEOUT -eq 0 ]; then
        echo "Timeout reached. No VDR file generated."
      fi
      shell: /usr/bin/bash -e {0}
      env:
        pythonLocation: /opt/hostedtoolcache/Python/3.10.14/x64
        PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.10.14/x64/lib/pkgconfig
        Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.10.14/x64
        Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.10.14/x64
        Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.10.14/x64
        LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.10.14/x64/lib
    ╭─────────────────────── Donate to the OWASP Foundation ───────────────────────╮
    │ OWASP foundation relies on donations to fund our projects.                   │
    │ Please donate at:                                                            │
    │ https://owasp.org/donate/?reponame=www-project-dep-scan&title=OWASP+depscan  │
    ╰──────────────────────────────────────────────────────────────────────────────╯
    INFO [2024-06-17 00:02:07,315] About to download the vulnerability database from ghcr.io/appthreat/vdbgz:v5. This might take a while ...
    INFO [2024-06-17 00:02:43,343] Performing regular scan for /home/runner/work/volttron-core/volttron-core using plugin bom
    
    ██████╗ ███████╗██████╗ ███████╗ ██████╗ █████╗ ███╗   ██╗
    ██╔══██╗██╔════╝██╔══██╗██╔════╝██╔════╝██╔══██╗████╗  ██║
    ██║  ██║█████╗  ██████╔╝███████╗██║     ███████║██╔██╗ ██║
    ██║  ██║██╔══╝  ██╔═══╝ ╚════██║██║     ██╔══██║██║╚██╗██║
    ██████╔╝███████╗██║     ███████║╚██████╗██║  ██║██║ ╚████║
    ╚═════╝ ╚══════╝╚═╝     ╚══════╝ ╚═════╝╚═╝  ╚═╝╚═╝  ╚═══╝
    
                             Dependency Scan Results (BOM)                          
    ╔══════════════════════╤══════════════════════╤═════════════╤══════════╤═══════╗
    ║ Dependency Tree      │ Insights             │ Fix Version │ Severity │ Score ║
    ╟──────────────────────┼──────────────────────┼─────────────┼──────────┼───────╢
    ║ volttron@latest      │ 📓 Indirect          │ 42.0.2      │ MEDIUM   │   5.5 ║
    ║ └── cryptography@36… │ dependency           │             │          │       ║
    ║     ⬅ CVE-2024-0727  │ 🧾 Vendor Confirmed  │             │          │       ║
    ╟──────────────────────┼──────────────────────┼─────────────┼──────────┼───────╢
    ║ volttron@latest      │ 📓 Indirect          │ 42.0.2      │ HIGH     │   7.5 ║
    ║ └── cryptography@36… │ dependency           │             │          │       ║
    ║     ⬅ CVE-2023-50782 │ 🧾 Vendor Confirmed  │             │          │       ║
    ╟──────────────────────┼──────────────────────┼─────────────┼──────────┼───────╢
    ║ volttron@latest      │ 📓 Indirect          │ 41.0.0      │ LOW      │   2.0 ║
    ║ └── cryptography@36… │ dependency           │             │          │       ║
    ║     ⬅                │                      │             │          │       ║
    ║     GHSA-5cpq-8wj7-… │                      │             │          │       ║
    ╟──────────────────────┼──────────────────────┼─────────────┼──────────┼───────╢
    ║ watchdog-gevent@0.1… │ 📓 Indirect          │ 23.9.0      │ CRITICAL │   9.8 ║
    ║ └── gevent@21.12.0 ⬅ │ dependency           │             │          │       ║
    ║     CVE-2023-41419   │                      │             │          │       ║
    ╟──────────────────────┼──────────────────────┼─────────────┼──────────┼───────╢
    ║ zope.interface@6.4.… │ 📓 Indirect          │ 65.5.1      │ MEDIUM   │   5.9 ║
    ║ └── setuptools@65.5… │ dependency           │             │          │       ║
    ║     ⬅ CVE-2022-40897 │ 🧾 Vendor Confirmed  │             │          │       ║
    ╟──────────────────────┼──────────────────────┼─────────────┼──────────┼───────╢
    ║ volttron@latest      │ 📓 Indirect          │ 41.0.4      │ LOW      │   2.0 ║
    ║ └── cryptography@36… │ dependency           │             │          │       ║
    ║     ⬅                │                      │             │          │       ║
    ║     GHSA-v8gr-m533-… │                      │             │          │       ║
    ╟──────────────────────┼──────────────────────┼─────────────┼──────────┼───────╢
    ║ volttron@latest      │ 📓 Indirect          │ 41.0.6      │ MEDIUM   │   5.9 ║
    ║ └── cryptography@36… │ dependency           │             │          │       ║
    ║     ⬅ CVE-2023-49083 │ 🧾 Vendor Confirmed  │             │          │       ║
    ╟──────────────────────┼──────────────────────┼─────────────┼──────────┼───────╢
    ║ volttron@latest      │ 📓 Indirect          │ 41.0.3      │ LOW      │   2.0 ║
    ║ └── cryptography@36… │ dependency           │             │          │       ║
    ║     ⬅                │                      │             │          │       ║
    ║     GHSA-jm77-qphf-… │                      │             │          │       ║
    ╟──────────────────────┼──────────────────────┼─────────────┼──────────┼───────╢
    ║ volttron@latest      │ 📓 Indirect          │ 39.0.1      │ HIGH     │   7.4 ║
    ║ └── cryptography@36… │ dependency           │             │          │       ║
    ║     ⬅ CVE-2023-0286  │ 🧾 Vendor Confirmed  │             │          │       ║
    ╟──────────────────────┼──────────────────────┼─────────────┼──────────┼───────╢
    ║ volttron@latest      │ 📓 Indirect          │ 39.0.1      │ MEDIUM   │   6.5 ║
    ║ └── cryptography@36… │ dependency           │             │          │       ║
    ║     ⬅ CVE-2023-23931 │                      │             │          │       ║
    ╚══════════════════════╧══════════════════════╧═════════════╧══════════╧═══════╝
    VDR file generated and moved to reports directory.

This approach ensures a robust, automated process for generating and maintaining SBOMs, enhancing security and compliance for the volttron-core repository.

kefeimo commented 5 months ago

Question:

How does the cdxgen -t python . command generate the BOM file? Specifically, does it retrieve dependency information from the virtual environment or the pyproject.toml file? Given that the cdxgen tool can obtain dependency information from various sources, what is the order of precedence for these sources?

Response:

The cdxgen -t python . command generates a Bill of Materials (BOM) file for a Python project by analyzing its dependencies. It can retrieve dependency information from several sources, following a specific precedence order:

  1. Primary Sources for Dependencies:
    • requirements.txt: Lists the exact versions of required packages and is given the highest precedence.
    • setup.py: Contains dependency specifications, often used in older Python projects.
    • Pipfile: Used by Pipenv for advanced dependency management.
    • pyproject.toml: The modern standard for Python projects, often used with tools like Poetry.
    • Virtual Environment: As a fallback, the tool can inspect the active virtual environment to determine installed packages and their versions.
  2. Precedence Order:
    • 1. requirements.txt: If present, this file is prioritized because it provides a clear list of dependencies.
    • 2. setup.py: Used if requirements.txt is not found.
    • 3. Pipfile: Checked next if neither requirements.txt nor setup.py are present.
    • 4. pyproject.toml: Used if none of the previous files are found.
    • 5. Virtual Environment: Inspected only if none of the specific dependency files are available, as this might include packages not explicitly required by the project.
  3. Process:
    • When the cdxgen command is run, it scans the project directory for the presence of these files.
    • It parses the file with the highest precedence found, extracts the list of dependencies along with their versions, and compiles this information into a BOM file.
    • The BOM file is typically in JSON or XML format and follows standards like CycloneDX to ensure compatibility with various tools and systems.

By following this precedence order, cdxgen ensures it accurately captures the project's dependencies, preferring explicit declarations in configuration files over inferred data from the virtual environment. This process results in a reliable BOM file that can be used for security analysis, compliance, and other purposes.