[Trufflehog Update] Deprecation of GitGuardian and mandatory update to Trufflehog

matbmoser commented 1 day ago

Description

The GitGuardian secret scanning tool licence is now going to be expired, therefore in order to maintain the Security of the Tractus-X Repositories there will be inforced the TRG-8.03 for all Tractus-X repos.

In order to keep the secret scanning functionality, it is required to add a workflow, so that before PRs are merged, there will be scanned for any API secrets, passwords, etc. Preventing you to publish into the open source repo main branch important secrets.

Why?

TruffleHog is an open source tool designed to identify sensitive information, such as API keys, passwords, and other credentials, that may have been inadvertently committed to your code repository. This tool is expected to be used in parallel to the native GitHub Secret Scanning tool.

Goal

Remove all GitGuardian workflows/documentation and substitute them to Trufflehog workflows, resolving all the vulnerabilities raised

TRG 8.03

Detecting and removing these secrets is crucial for maintaining the security of your application and infrastructure. TruffleHog performs a thorough search by checking the entire repository history, not just the latest commits. This means it can find secrets that were committed in the past and might still pose a security risk.

Configure your GitHub Actions to include:

workflow dispatch: Manual workflow execution. schedule: Schedule the workflow to run at least once a week with 0 0 0. push and pull_request: Activate the workflow on both push and pull request events targeting the branch that contains the code for the currently supported version, which may not necessarily be the main branch. This is the branch from which new releases will be made. Note: extra_args: --filter-entropy=4 --results=verified,unknown

Including extra_args: --filter-entropy=4 --results=verified,unknown in the GitHub Actions workflow ensures that TruffleHog focuses on detecting high-entropy strings, which are more likely to be sensitive information such as passwords or API keys. This setup also instructs TruffleHog to report both verified secrets and potential but unverified secrets, providing a comprehensive security scan that helps identify and address all possible vulnerabilities in the code.

Including run: exit 1 in a step of a GitHub Actions workflow, as demonstrated below, commands the workflow to halt execution. This ensures that should TruffleHog uncover any secrets during its scan, the workflow promptly terminates in failure.

GitHub Actions allows you to define workflows to automatically run TruffleHog scans on your code. You'll see the output that triggered the failure directly in the logs.

Here’s how you can set it up:

name: "TruffleHog"

on:
  push:
    branches: ["main"]
  pull_request:
    # The branches below must be a subset of the branches above
    branches: ["main"]
  schedule:
    - cron: "0 0 * * *" # Once a day
  workflow_dispatch:

permissions:
  actions: read
  contents: read
  security-events: write
  id-token: write
  issues: write

jobs:
  ScanSecrets:
    name: Scan secrets
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Ensure full clone for pull request workflows

      - name: TruffleHog OSS
        id: trufflehog
        uses: trufflesecurity/trufflehog@7e78ca385fb82c19568c7a4b341c97d57d9aa5e1
        continue-on-error: true
        with:
          path: ./  # Scan the entire repository
          base: "${{ github.event.repository.default_branch }}"  # Set base branch for comparison (pull requests)
          extra_args: --filter-entropy=4 --results=verified,unknown --debug

      - name: Scan Results Status
        if: steps.trufflehog.outcome == 'failure'
        run: exit 1  # Set workflow run to failure if TruffleHog finds secrets

I have runned a scan https://github.com/eclipse-tractusx/sig-infra/pull/545 for checking which repos have the Trufflehog and which ones not:

eclipse-tractusx/digital-product-pass contains .github/workflows/trufflehog.yaml
eclipse-tractusx/tractusx-edc contains .github/workflows/secrets-scan.yml
eclipse-tractusx/portal-backend contains .github/workflows/trufflehog.yml
eclipse-tractusx/portal contains .github/workflows/trufflehog.yml
eclipse-tractusx/puris contains .github/workflows/trufflehog.yml
eclipse-tractusx/portal-frontend contains .github/workflows/trufflehog.yml
eclipse-tractusx/ssi-dim-wallet-stub contains .github/workflows/trufflehog.yml
eclipse-tractusx/portal-iam contains .github/workflows/trufflehog.yml
eclipse-tractusx/portal-assets contains .github/workflows/trufflehog.yml
eclipse-tractusx/ssi-credential-issuer contains .github/workflows/trufflehog.yml
eclipse-tractusx/ssi-authority-schema-registry contains .github/workflows/trufflehog.yml
eclipse-tractusx/portal-frontend-registration contains .github/workflows/trufflehog.yml
eclipse-tractusx/policy-hub contains .github/workflows/trufflehog.yml
eclipse-tractusx/bpdm contains .github/workflows/trufflehog.yml
eclipse-tractusx/managed-service-orchestrator contains .github/workflows/trufflehog.yml
eclipse-tractusx/sldt-bpn-discovery contains .github/workflows/trufflehog.yml
eclipse-tractusx/sldt-discovery-finder contains .github/workflows/trufflehog.yml
eclipse-tractusx/sldt-semantic-hub contains .github/workflows/trufflehog.yml
eclipse-tractusx/sldt-digital-twin-registry contains .github/workflows/trufflehog.yml
eclipse-tractusx/demand-capacity-mgmt contains .github/workflows/trufflehog.yml

List of Repositories that do not contain Trufflehog (archived are excluded):

eclipse-tractusx/eclipse-tractusx.github.io
eclipse-tractusx/.eclipsefdn
eclipse-tractusx/api-hub
eclipse-tractusx/charts
eclipse-tractusx/sig-release
eclipse-tractusx/portal-shared-components
eclipse-tractusx/tutorial-resources
eclipse-tractusx/tractusx-edc-template
eclipse-tractusx/bpn-did-resolution-service
eclipse-tractusx/sldt-semantic-models
eclipse-tractusx/knowledge-agents-aas-bridge
eclipse-tractusx/managed-simple-data-exchanger-frontend
eclipse-tractusx/traceability-foss
eclipse-tractusx/managed-identity-wallet
eclipse-tractusx/item-relationship-service
eclipse-tractusx/sd-factory
eclipse-tractusx/data-exchange-test-service
eclipse-tractusx/knowledge-agents
eclipse-tractusx/knowledge-agents-edc
eclipse-tractusx/tractus-x-umbrella
eclipse-tractusx/vas-country-risk
eclipse-tractusx/sldt-ontology-model
eclipse-tractusx/sig-security
eclipse-tractusx/tractus-x-release
eclipse-tractusx/managed-simple-data-exchanger-backend
eclipse-tractusx/sig-infra
eclipse-tractusx/managed-simple-data-exchanger
eclipse-tractusx/.github
eclipse-tractusx/SSI-agent-lib
eclipse-tractusx/eclipse-tractusx.github.io.largefiles
eclipse-tractusx/testdata-provider
eclipse-tractusx/tractusx-profiles
eclipse-tractusx/app-dashboard

Since the gitguardian do not needed a worflow but was there by the default, now we need to include the workflows in order to keep the secret scanning functionality.

matbmoser commented 1 day ago

I have runned a scan https://github.com/eclipse-tractusx/sig-infra/pull/545 for checking which repos have the Trufflehog and which ones not:

eclipse-tractusx/digital-product-pass contains .github/workflows/trufflehog.yaml
eclipse-tractusx/tractusx-edc contains .github/workflows/secrets-scan.yml
eclipse-tractusx/portal-backend contains .github/workflows/trufflehog.yml
eclipse-tractusx/portal contains .github/workflows/trufflehog.yml
eclipse-tractusx/puris contains .github/workflows/trufflehog.yml
eclipse-tractusx/portal-frontend contains .github/workflows/trufflehog.yml
eclipse-tractusx/ssi-dim-wallet-stub contains .github/workflows/trufflehog.yml
eclipse-tractusx/portal-iam contains .github/workflows/trufflehog.yml
eclipse-tractusx/portal-assets contains .github/workflows/trufflehog.yml
eclipse-tractusx/ssi-credential-issuer contains .github/workflows/trufflehog.yml
eclipse-tractusx/ssi-authority-schema-registry contains .github/workflows/trufflehog.yml
eclipse-tractusx/portal-frontend-registration contains .github/workflows/trufflehog.yml
eclipse-tractusx/policy-hub contains .github/workflows/trufflehog.yml
eclipse-tractusx/bpdm contains .github/workflows/trufflehog.yml
eclipse-tractusx/managed-service-orchestrator contains .github/workflows/trufflehog.yml
eclipse-tractusx/sldt-bpn-discovery contains .github/workflows/trufflehog.yml
eclipse-tractusx/sldt-discovery-finder contains .github/workflows/trufflehog.yml
eclipse-tractusx/sldt-semantic-hub contains .github/workflows/trufflehog.yml
eclipse-tractusx/sldt-digital-twin-registry contains .github/workflows/trufflehog.yml
eclipse-tractusx/demand-capacity-mgmt contains .github/workflows/trufflehog.yml

List of Repositories that do not contain Trufflehog (archived are excluded):

eclipse-tractusx/eclipse-tractusx.github.io
eclipse-tractusx/.eclipsefdn
eclipse-tractusx/api-hub
eclipse-tractusx/charts
eclipse-tractusx/sig-release
eclipse-tractusx/portal-shared-components
eclipse-tractusx/tutorial-resources
eclipse-tractusx/tractusx-edc-template
eclipse-tractusx/bpn-did-resolution-service
eclipse-tractusx/sldt-semantic-models
eclipse-tractusx/knowledge-agents-aas-bridge
eclipse-tractusx/managed-simple-data-exchanger-frontend
eclipse-tractusx/traceability-foss
eclipse-tractusx/managed-identity-wallet
eclipse-tractusx/item-relationship-service
eclipse-tractusx/sd-factory
eclipse-tractusx/data-exchange-test-service
eclipse-tractusx/knowledge-agents
eclipse-tractusx/knowledge-agents-edc
eclipse-tractusx/tractus-x-umbrella
eclipse-tractusx/vas-country-risk
eclipse-tractusx/sldt-ontology-model
eclipse-tractusx/sig-security
eclipse-tractusx/tractus-x-release
eclipse-tractusx/managed-simple-data-exchanger-backend
eclipse-tractusx/sig-infra
eclipse-tractusx/managed-simple-data-exchanger
eclipse-tractusx/.github
eclipse-tractusx/SSI-agent-lib
eclipse-tractusx/eclipse-tractusx.github.io.largefiles
eclipse-tractusx/testdata-provider
eclipse-tractusx/tractusx-profiles
eclipse-tractusx/app-dashboard

Since the gitguardian do not needed a worflow but was there by the default, now we need to include the workflows in order to keep the secret scanning functionality.

netomi commented 1 day ago

fyi: secret scanning and secret scanning push protection is already enabled for all repos in the eclipse-tractusx organization and currently all committers are setup as security managers so can access information in the security tab of the organization / repo to access whether any secrets have been detected (push protection is not able to prevent all secrets from being pushed). So thats a good start, having a second way to do secret scanning is certainly a good idea.

However, may I suggest that you add the template e.g. to the sig-security repo so that the other repos just have to call that workflow instead of copy & pasting the whole workflow again and again? This would be similar to how the mavenLicenseCheck.yml is setup as you can see here: https://github.com/eclipse-dash/dash-licenses/blob/master/.github/workflows/mavenLicenseCheck.yml

That way you can adjust that easily in one place without having to check it for every repo if something has to change. Also I would suggest to pin the action to a specific version and update as needed.

netomi commented 1 day ago

btw the overview here https://entro.security/blog/securing-the-code-navigating-code-and-github-secrets-scanning/ does not talk so well about trufflehog due to its high number of false positives, but I guess you did some research beforehand, I actually never used that tool.

matbmoser commented 21 hours ago

Hi @netomi, thank you for your support 💯

The Trufflehog tool was selected by our security experts as the preferred open source security scan tool to substitute GitGuardian. There was already a researched done by them previously and also the repositories that I have mentioned have already implemented the workflow and tested.

Happy to know that we have enabled by default the secret scanning from the GitHub side! The reason we are using an additional scan is to double check in the PRs if there is any secret there.

I am aware it can raise several false positives, but the same was happening with GitGuardian if I am honest... Good point! It's a good idea to place the Trufflehog workflow in one of our "main" repositories for security. I will also point the trufflehog version to 7e78ca385fb82c19568c7a4b341c97d57d9aa5e1

I will bring this topic and your comments in our next office hour and committer meeting. @evegufy and @stephanbcbauer lets this discuss this further on 💯

eclipse-tractusx / sig-security