What analysis was done on the threats posed by a " Project Security Metrics"

[cyberscribbleorg]

We are introducing a prospective reference tool for customers/developers to refer to, in this sense, what kind of security threat/risk analysis are done on introducing such tool and providing a reference go-to place in to open-source landscape ?

• Does this tool introduces new threats and increases customer compromise level ?
• Does this tool introduces new type of attack vectors ?

I went through your documentation and meeting minutes, I did not see any such analysis for me to refer to and what type of counter measures or protections are put in place.

PS : My native mother tongue is not English, Please pardon my self if I misspoke some technical Jargons, I know when it comes to security people are peculiar about technological jargons. but I hope the message is received well, If not, Please comment on the ticket , I will try my best to clarify within my reach.

[scovetta]

Hi @pjeyarat! If I understand correctly, you're talking about whether this tool itself could be used as part of an attack (meaning, the tool, dashboard, etc.) might have a flaw where an attacker could insert some bad data in (either at a technical vulnerability level like XSS or a higher-level attack like manipulating certain projects to get a "good" or "bad" standing)? Like maybe if we weight the number of commits too high in the scoring mechanism, an attacker could create a lot of bogus commits that would elevate their project above others?

Or did you mean that creating such a tool would have others rely on it, and trust what it says -- and make decisions based on that trust -- and if we made a mistake and the customer was attacked as a result -- would they come back to us?

To address these, I think:

• Technical Attacks: We need to test/validate that our code is secure generally.
• Defend Against Manipulation: We need to be transparent about the criteria we use, especially for aggregations, and take feedback/input from the community. We should also specifically threat model this aspect of the system.
• Mistakes and Trust: Transparency should go some way here, but this is also what separates this project from a "certification" body -- we're making no claims about whether a user should rely on X, but rather giving them information upon which to make such a decision. We could give them wrong information (a bug), but I think we can mitigate this by having multiple individuals be involved in the implementation.

I certainly welcome others' thoughts, too on this!

[cyberscribbleorg]

Dear @scovetta

In my opinion, before we embark on a Project associated to security we need to produce a "Threat/Vulnerability/Risk) assessment of such Project introduction in to the public domain, For example a stake-holder based risk analysis should include;

• Risk for customers
• Risk for developers
• Risk for the public
• Risk for the Project Participants ( OSSF/Project-Security-Metrics) team
• Risk for project enablers ( People who will supply data and tools to enable our Project-Security-Matrix eco-system, Linux Foundation / OSSF etc )

Additionally, We need to understand the type of attack vectors of our Project ( Here are some over the top of my head, this requires careful analysis) introduces,

• Attacks using the end product ( tool) ( What you have mentioned in above as Technical Attack )
• Attacks using the tool operational process ( Somewhat similar to what you have mentioned as manipulation )
• Attacks using the tool policy ( Short term/ Long Term )
• Attacks using the Trust of OSSF / Linux Foundation within the mind of general public/customers and developers

If we have already done this, Please point to the documentation which analyzes and concludes that " Project Security Metrics" is indeed a worthwhile endeavor to undertake upon.

Finally, we need to analyze that this Project does not violate the "Ethics", "Logic" or "Emotional" aspect of stake-holders and participants. If it does so, we need to improve the process to find a win-win situation to all parties associated to the eco-system.

Please guide me in the right direction If I have missed or under looked or over-looked any aspect on this,.

I welcome others and experts thoughts/inputs on this!

[scovetta]

Hi @pjeyarat! I appreciate your thoughts here -- we don't have anything formally documented like this. Would you be interested in driving this effort to create such a document? I think it would be very useful.

[cyberscribbleorg]

Dear @scovetta ,

Thank you very much for your response and offer. I would like to humbly notify you that, Such a documentation will not be complete with ONLY an individual effort but rather requires a collective effort with inputs from diversity. I am glad to take participate in such effort if We can collectively discuss and finalize such an effort.

If it sounds, worth endeavoring upon for yourself, We may be able to discuss this on our meeting and see whether we could collectively pen such a documentation.