search

ossf / scorecard

OpenSSF Scorecard - Security health metrics for Open Source
https://scorecard.dev
Apache License 2.0
4.3k stars 470 forks source link

BUG: Security-Policy doesn't recognize asciidoc files #1347

Closed georgettica closed 2 years ago

georgettica commented 2 years ago

Describe the bug when running the tool on my repo https://github.com/georgettica/venv I found it doesn't find my SECURITY.adoc file

Reproduction steps docker run -e GITHUB_AUTH_TOKEN=XXXX gcr.io/openssf/scorecard:stable --show-details --repo=https://github.com/georgettica/venv

Expected behavior that this check will pass

Additional context I have been cleaning this repo for this evening, so the latest commits are regarding making it comply with scorecard

laurentsimon commented 2 years ago

cc @david-a-wheeler

laurentsimon commented 2 years ago

Additional context I have been cleaning this repo for this evening, so the latest commits are regarding making it comply with scorecard

awesome! Were they pain points using the tool? Note that we also have a GitHub action we're in beta, in case you want to give it a try https://github.com/ossf/scorecard/issues/1074#issuecomment-933653061

georgettica commented 2 years ago

@laurentsimon I had some:

I'll see if I can recall others aswell

georgettica commented 2 years ago

Last thing:

laurentsimon commented 2 years ago

Describe the bug when running the tool on my repo https://github.com/georgettica/venv I found it doesn't find my SECURITY.adoc file

Can you tell us why you use an adoc rather than .md? Currently we have support for .md, which seems to be an md https://docs.github.com/en/code-security/getting-started/adding-a-security-policy-to-your-repository

laurentsimon commented 2 years ago

@laurentsimon I had some:

  • I sometimes got warning and info messages, although I did things right

info messages mean something you did is good; warnings means something bad. Is this something we should explicitly write about in the README?

  • once I got alot of data in the output field, the table display was problematical (moved to json to solve)

maybe we could make the JSON format the default, you're not the first to complain about the table.

  • version-pinning is still not clear to me (I version pinned everything I saw, but it still complains

we recommend pinning by hash, not by version. Can you point to the docs you read and where it was confusing?

  • when setting the permissions stanza, the github docs + the scorecard docs didn't emphasize the importance of putting in the top level for the full points

I'll add this to the doc, you're right it's not well documented, thanks!

I'll see if I can recall others aswell

cc @olivekl

laurentsimon commented 2 years ago

follow

@laurentsimon I had some:

  • I sometimes got warning and info messages, although I did things right

info messages mean something you did is good; warnings means something bad. Is this something we should explicitly write about in the README?

  • once I got alot of data in the output field, the table display was problematical (moved to json to solve)

maybe we could make the JSON format the default, you're not the first to complain about the table.

  • version-pinning is still not clear to me (I version pinned everything I saw, but it still complains

we recommend pinning by hash, not by version. Can you point to the docs you read and where it was confusing?

  • when setting the permissions stanza, the github docs + the scorecard docs didn't emphasize the importance of putting in the top level for the full points

I'll add this to the doc, you're right it's not well documented, thanks!

actually our doc already says The highest score is awarded when the permissions definitions in each workflow's yaml file are set as read-only at the [top level](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#permissions) and the required write permissions are declared at the [run-level](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idpermissions).

georgettica commented 2 years ago

I would try and separate each good or bad in a different section AND hide bad checks if they are not valid anymore

I wouldn't say formatting in json the the b st way, but that the table output can be better displaying the array of results.

Do you mean we should pin the.githib actions by hash?

The doc change I wanted was of githubs, but I would be glad if the error message about permissions would point to the top level thing. The docs I am referring to are the github docs you are pointing to, not your own

georgettica commented 2 years ago

And I agreed that github is advocating markdown, but I think that keeping asciidoc is a valid request

laurentsimon commented 2 years ago

@chrismcgehee @oliverchang @azeemsgoogle @naveensrinivasan would anyone object to adding the adoc extension to the readme check support?

naveensrinivasan commented 2 years ago

I don't see an issue. Let's do it.

azeemshaikh38 commented 2 years ago

SGTM.

david-a-wheeler commented 2 years ago

I'd go further, we should support asciidoc. Markdown wasn't handed down from the gods, we should meet projects where they are.

georgettica commented 2 years ago

Yes! Thanks 🙏