ossf / scorecard

OpenSSF Scorecard - Security health metrics for Open Source
https://scorecard.dev
Apache License 2.0
4.47k stars 489 forks source link

"Pinned-Dependencies" shouldn't complain about known GHActions that can't be pinned to any specific version #1305

Closed evverx closed 2 years ago

evverx commented 2 years ago

This is mostly inspired by https://github.com/google/oss-fuzz/issues/6836

I agree with @jonathanmetzman that it doesn't make much sense to point CIFuzz to anything other than the master branch (in its current form at least) and I'm not exactly sure how exactly CIFuzz is supposed to be versioned to let Dependabot handle it automatically but until it's solved I don't think scorecard should complain about it. It's not my fault after all. All I wanted is to fuzz the code on PRs a bit :-)

evverx commented 2 years ago

FWIW I think CIFuzz should be added to the Fuzzing check as well and projects using both OSS-Fuzz and CIFuzz should get higher scores

laurentsimon commented 2 years ago

agreed on CIFuzz, and clusterfuzzlite too. @oliverchang is working on the clusterfuzzlite PR. Can you create a tracking issue for CIFuzz?

evverx commented 2 years ago

@laurentsimon I'm not sure about ClusterFuzzLite but I think CIFuzz is a bit tricky. It doesn't support forks https://github.com/google/oss-fuzz/issues/3731 so systemd for example just turns it off with github.repository == 'systemd/systemd' so it seems the check should look for PRs where it's really run instead of just looking for use cifuzz in .github/workflows/*

laurentsimon commented 2 years ago

Gotcha. Feel free to create a PR. cc @oliverchang from OSS-Fuzz for comments

evverx commented 2 years ago

Feel free to create a PR.

FWIW I don't think I'm going to create PRs anymore because having received what I would call condescending comments in one of issues I've opened in one of OSSF projects I don't want my account to be associated with OSSF in any way. I think if would be great if the project could be moved to the "google" organization on GitHub (but I understand that it isn't possible). I'm going to keep commenting on issues here though since GitHub doesn't link this activity with organizations.

laurentsimon commented 2 years ago

Hey @evverx I'm sorry to hear that. I really am. Your input, feedback and PRs on scorecard have just been incredibly useful. I would really like you to continue to contribute: you've made scorecard so much better already!

I am 100% certain I speak for the whole team here.

If you don't want to discuss this in the issue, I understand. I'm open to chat/DM slack on the OSSF channel. Any other means works, please let me know. Can start by email if it's more convenient.

We cannot improve unless we identify the root cause of the problem.

evverx commented 2 years ago

@laurentsimon I wrote what I thought in that issue so I'd just leave it at that. As far as I can tell, OSSF (as a whole) already received that kind of feedback about a year ago and as far as I can see nothing has changed since then so it seems it isn't a priority there. Even if it was I'm not sure I understand what the point of OSSF is considering it isn't actually responsible for anything, can't communicate with open source developers, can't manage and run programs Google has been run for I don't know how many years and so on.

Just to clarify, I have absolutely no problem with projects like scorecard that I think are helpful. It's been a pleasure to work with.

naveensrinivasan commented 2 years ago

@evverx I am sorry to hear that. We appreciate your inputs which have been extremely helpful for the project.

evverx commented 2 years ago

@naveensrinivasan thanks!

It isn't the end of the world though :-) I'm thick-skinned anyway :-)

To be fair I'm not a saint either (but in my defence I don't represent any organizations officially here on GitHub so the rules I follow are a bit more relaxed)

azeemshaikh38 commented 2 years ago

That's sad to hear @evverx. Hope to see your continued involvement in Scorecard (and potentially other OpenSSF projects). Do let us know if there's anything we can do to make things better in the meanwhile.

inferno-chromium commented 2 years ago

@evverx - you embody the true spirit of open source community, and we have learnt from your experience and passion to make open source more secure (in several places, OSS-Fuzz, ClusterFuzzLite and scorecard to name a few). I saw some of your frustation on the mfa project, but I can assure you we can try to resolve it offline (and it is not OpenSSF or its member fault). We are all working very hard to create security tools for benefit of the community. So, your contributions in any form (PR or comments) will always help, I hope you please reconsider your decision.

evverx commented 2 years ago

@inferno-chromium I appreciate the OSS-Fuzz team and what it has been doing for open source projects. In my opinion it's one of the best examples of what projects that communicate with open source developers should look like. I keep contributing there and today I've already opened 2 issues and 1 PR :-) And there (or anywhere else apart from the mfa project really) I have absolutely no problem with my ideas (some of which I have to admit don't make much sense sometimes) being rejected or questions not being answered (though I hope https://github.com/google/clusterfuzzlite/issues/64 can be reconsidered :-)). But I can't imagine a scenario where I could be told "it's done how it's done. say thank you. you're free to complain elsewhere though" there and I'm afraid I simply can't accept this kind of patronizing.

My comments there were too harsh though and I apologize for that.

evverx commented 2 years ago

Looks like the issue can be closed. As far as I understand it should be addressed by analogy with the "binary-artifacts" check: https://github.com/ossf/scorecard/issues/1256#issuecomment-1000091579