search

ossf / ai-ml-security

Potential WG on Artificial Intelligence and Machine Learning (AI/ML)
Apache License 2.0
53 stars 9 forks source link

Map AIML WG outputs to MLSecOps diagram #16

Open sevansdell opened 4 months ago

sevansdell commented 4 months ago

I really like the MLSecOps document shared by Ericson: https://www.ericsson.com/en/reports-and-papers/white-papers/mlsecops-protecting-the-ai-ml-lifecycle-in-telecom

  1. I would like to show where in the MLSecOps lifecycle security artifacts/artifact checking helps improve security.
  2. I would like to map how OWASP ML top 10 are mitigated using MLSecOps in the same diagram https://owasp.org/www-project-machine-learning-security-top-10/#:~:text=Top%2010%20Machine%20Learning%20Security%20Risks%201%20ML01%3A2023,Learning%20Attack%208%20ML08%3A2023%20Model%20Skewing%20More%20items.
  3. I would like to identify where open source or closed source data, models and code impact the AI supply chain/ ML Lifecycle.

I would like to discuss in a future call if the team feels this is an interesting visual/written output on which to collaborate, if is already duplicating an existing industry effort, or if it's a good idea but doesn't fall into the scope of the AIML WG.

TheFoxAtWork commented 4 months ago

Yes! This is along the lines of my ask on the call yesterday. I believe @camaleon2016 had mentioned working on something related in another group and would report back/share.

Jay - is this the same or different than what you mentioned on the call?

camaleon2016 commented 4 months ago

Yes! This is along the lines of my ask on the call yesterday. I believe @camaleon2016 had mentioned working on something related in another group and would report back/share.

Jay - is this the same or different than what you mentioned on the call?

This is different but equally good!

sevansdell commented 3 months ago

Have started work on this. Will share a v1 in a couple weeks with the team.

ashxz47 commented 3 months ago

Concerning point 1. Could you please clarify what you mean? I guess the artifact security checking is continuous through the lifecycle, highlighted in the figure by the green boxes and described in the text. For example, you should employ proper security measures if you have to get the data from an untrusted source. Or encrypt and integrity protect artifacts in transit and at rest.

Concerning 3. The diagram is quite straightforward on the typical lifecycle, but you might have quite a lot of issues when doing the transfer learning somewhere in the middle. I think such external data and foundational models are the biggest threat as they are too big and opaque. I think it would be nice to add the roles/responsibilities to the figure. As well it would also be nice to present a maturity model, as implementing such full MLSecOps is not easy.

sevansdell commented 1 month ago

The Ericsson team may join the 10/14 meeting as they can, and we would like to share the first draft October 28. The draft will be is open to team feedback prior to finalizing v1

sevansdell commented 1 month ago

Concerning point 1. Could you please clarify what you mean? I guess the artifact security checking is continuous through the lifecycle, highlighted in the figure by the green boxes and described in the text. For example, you should employ proper security measures if you have to get the data from an untrusted source. Or encrypt and integrity protect artifacts in transit and at rest.

Concerning 3. The diagram is quite straightforward on the typical lifecycle, but you might have quite a lot of issues when doing the transfer learning somewhere in the middle. I think such external data and foundational models are the biggest threat as they are too big and opaque. I think it would be nice to add the roles/responsibilities to the figure. As well it would also be nice to present a maturity model, as implementing such full MLSecOps is not easy.

Andrey with Ericsson will be sharing the initial diagram in our AIML WG meeting Oct 24. We will all have a chance to collaborate on the initial document together asynch.

sevansdell commented 1 month ago

Using this comment to capture some future work associated with this reference architecture:

-to make an LLMSecOps version of this diagram -map both versions to how the diagrams prevent OWASP top 10. -documentation for how supply chain security practices that help prevent the OWASP top10 can be shared (e.g. model scanning for malware prior to containerizing and sharing for use) -map how [OpenSSF] supply chain tools close gaps (or don’t close gaps and we could see if there is an effort needed in OpenSSF to close it) -create implementation guides that map to CSA AI risk/threat model white papers -Include indicators of where new AI component-specific supply chain tools like the Model Signing SIG fit into the architectures -map to various regulatory frameworks/schemas, like the NIST AI RMF and NIST SSDF. Be used as a talking point for evolving software security frameworks to extend to AIML.

In short, the goal is to apply lessons learned in OpenSSF from DevOps becoming DevSecOps and retrofitting the software supply chain to help avoid the same pain in new systems with emerging AIML workloads is the challenge of our time. We have a window today to change the industry by showing how to put the Sec in MLSecOps and LLMSecOps and proactively close gaps in the AIML supply chain.

abdullahgarcia commented 2 weeks ago

@sevansdell hi!

Are there separate sessions besides the working group meetings to work on this? How do I get involved?

Thanks.