ACCEPTED - LLM10: Consider adding "Use a centralized ML Model Inventory or Registry" as a Prevention under LLM10

[guerilla7]

Hi Team,

Congrats on the first iteration!

I was hunting for either a "registry" or "inventory" word on the document and could not find one.

Based on my experience in threat modeling internally-developed ML/LLM applications, development teams are prone to storing the actual ML model or bits and pieces of related artifacts (documents, code, and other data that can be used to understand the purpose of the model) on their local machine, personal code repo, etc., contributing to LLM10 - Model Theft.

Under Prevention, I would like to propose adding the following entry:

• Use a centralized ML Model Inventory or Registry for ML models used in production. Having a centralized model registry prevents unauthorized access to ML Models via access controls, authentication, and monitoring/logging capability which are good foundations for governance. Having a centralized repository is also beneficial for collecting data about algorithms used by the models for the purposes of compliance, risk assessments, and risk mitigation.

Examples of centralized ML model registries include MLFlow, Iterative.ai, GCP Vertex AI Model Registry, and AWS SageMaker Model Registry.

References:

1. "The Current State of AI Governance" Report by Babl AI, Inc. & The Algorithmic Bias Lab
2. " Model Management section - Machine Learning for High-Risk Applications: Techniques for Responsible AI" by Hall, Curtis, & Pandey. O'Reilly Press April 2023.
3. AML.TA0000 ML Model Access - MITRE ATLAS™ (Adversarial Threat Landscape for Artificial-Intelligence Systems) ATT&CK Framework

[GangGreenTemperTatum]

Thank you! Many thanks for reaching out to us also.

Great suggestion, I will get this enhacement added to the next round and ACK back shortly.

Examples of centralized ML model registries include MLFlow, Iterative.ai, GCP Vertex AI Model Registry, and AWS SageMaker Model Registry.

I will stray away from this, as we do not want to directly reference vendors, models or make any distinctions that could lead towards bias

AML.TA0000 ML Model Access - MITRE ATLAS™ (Adversarial Threat Landscape for Artificial-Intelligence Systems) ATT&CK Framework

Happy to include this as a reference