score meaning is not right in document

gyliu513 commented 2 years ago

Describe the bug A clear and concise description of what the bug is.

Reproduction steps From https://github.com/ossf/scorecard#scoring

I think the above info is not right, as higher values means less risk, so 10 means no risk

Expected behavior A clear and concise description of what you expected to happen.

Additional context Add any other context about the problem here.

gyliu513 commented 2 years ago

Should not it be as following?

“Critical” risk checks are weighted at 0
“High” risk checks are weighted at 2.5
“Medium” risk checks are weighted at 5
“Low” risk checks are weighted at 7.5

gyliu513 commented 2 years ago

This is the score that I get from open-telemetry, from the output, I think high score means less risk.

% docker run -e GITHUB_AUTH_TOKEN=$GITHUB_AUTH_TOKEN gcr.io/openssf/scorecard:stable --repo=https://github.com/open-telemetry/opentelemetry-collector --format=json | jq .
{
  "date": "2022-07-12",
  "repo": {
    "name": "github.com/open-telemetry/opentelemetry-collector",
    "commit": "6133c820fd50d66585cec422738d116a9a22bad8"
  },
  "scorecard": {
    "version": "v4.4.0-11-g48291a3",
    "commit": "48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e"
  },
  "score": 6.6,
  "checks": [
    {
      "details": null,
      "score": 10,
      "reason": "no binaries found in the repo",
      "name": "Binary-Artifacts",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#binary-artifacts",
        "short": "Determines if the project has generated executable (binary) artifacts in the source repository."
      }
    },
    {
      "details": null,
      "score": 8,
      "reason": "branch protection is not maximal on development and all release branches",
      "name": "Branch-Protection",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#branch-protection",
        "short": "Determines if the default and release branches are protected with GitHub's branch protection settings."
      }
    },
    {
      "details": null,
      "score": 10,
      "reason": "30 out of 30 merged PRs checked by a CI test -- score normalized to 10",
      "name": "CI-Tests",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#ci-tests",
        "short": "Determines if the project runs tests before pull requests are merged."
      }
    },
    {
      "details": null,
      "score": 0,
      "reason": "no badge detected",
      "name": "CII-Best-Practices",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#cii-best-practices",
        "short": "Determines if the project has a CII Best Practices Badge."
      }
    },
    {
      "details": null,
      "score": 10,
      "reason": "all last 30 commits are reviewed through GitHub",
      "name": "Code-Review",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#code-review",
        "short": "Determines if the project requires code review before pull requests (aka merge requests) are merged."
      }
    },
    {
      "details": null,
      "score": 10,
      "reason": "38 different organizations found -- score normalized to 10",
      "name": "Contributors",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#contributors",
        "short": "Determines if the project has a set of contributors from multiple organizations (e.g., companies)."
      }
    },
    {
      "details": null,
      "score": 0,
      "reason": "dangerous workflow patterns detected",
      "name": "Dangerous-Workflow",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#dangerous-workflow",
        "short": "Determines if the project's GitHub Action workflows avoid dangerous patterns."
      }
    },
    {
      "details": null,
      "score": 10,
      "reason": "update tool detected",
      "name": "Dependency-Update-Tool",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#dependency-update-tool",
        "short": "Determines if the project uses a dependency update tool."
      }
    },
    {
      "details": null,
      "score": 0,
      "reason": "project is not fuzzed",
      "name": "Fuzzing",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#fuzzing",
        "short": "Determines if the project uses fuzzing."
      }
    },
    {
      "details": null,
      "score": 10,
      "reason": "license file detected",
      "name": "License",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#license",
        "short": "Determines if the project has defined a license."
      }
    },
    {
      "details": null,
      "score": 10,
      "reason": "30 commit(s) out of 30 and 26 issue activity out of 30 found in the last 90 days -- score normalized to 10",
      "name": "Maintained",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#maintained",
        "short": "Determines if the project is \"actively maintained\"."
      }
    },
    {
      "details": null,
      "score": 10,
      "reason": "publishing workflow detected",
      "name": "Packaging",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#packaging",
        "short": "Determines if the project is published as a package that others can easily download, install, easily update, and uninstall."
      }
    },
    {
      "details": null,
      "score": 7,
      "reason": "dependency not pinned by hash detected -- score normalized to 7",
      "name": "Pinned-Dependencies",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#pinned-dependencies",
        "short": "Determines if the project has declared and pinned its dependencies."
      }
    },
    {
      "details": null,
      "score": 10,
      "reason": "SAST tool is run on all commits",
      "name": "SAST",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#sast",
        "short": "Determines if the project uses static code analysis."
      }
    },
    {
      "details": null,
      "score": 10,
      "reason": "security policy file detected",
      "name": "Security-Policy",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#security-policy",
        "short": "Determines if the project has published a security policy."
      }
    },
    {
      "details": null,
      "score": 0,
      "reason": "0 out of 5 artifacts are signed -- score normalized to 0",
      "name": "Signed-Releases",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#signed-releases",
        "short": "Determines if the project cryptographically signs release artifacts."
      }
    },
    {
      "details": null,
      "score": 0,
      "reason": "non read-only tokens detected in GitHub workflows",
      "name": "Token-Permissions",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#token-permissions",
        "short": "Determines if the project's workflows follow the principle of least privilege."
      }
    },
    {
      "details": null,
      "score": 10,
      "reason": "no vulnerabilities detected",
      "name": "Vulnerabilities",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#vulnerabilities",
        "short": "Determines if the project has open, known unfixed vulnerabilities."
      }
    },
    {
      "details": null,
      "score": -1,
      "reason": "check is not supported for this request: SCORECARD_V6 is not set, not running the Webhook check",
      "name": "Webhooks",
      "documentation": {
        "url": "https://github.com/ossf/scorecard/blob/48291a3dd48a01b8c601f3f38f86f7c1f61d9e3e/docs/checks.md#webhooks",
        "short": "This check validate if the webhook defined in the repository have a token configured."
      }
    }
  ],
  "metadata": null
}

laurentsimon commented 2 years ago

score is how well the repo performs to mitigate risks: the higher the score, the better; so the lower the risk. Let us know if there's a better way to phrase it in the doc.

gyliu513 commented 2 years ago

@laurentsimon how about using https://github.com/ossf/scorecard/issues/2041#issuecomment-1181770383 to fix

https://github.com/ossf/scorecard#scoring

ossf / scorecard

score meaning is not right in document #2041