apache / polaris

Apache Polaris, the interoperable, open source catalog for Apache Iceberg
https://polaris.apache.org/
Apache License 2.0
1.13k stars 124 forks source link

[FEATURE REQUEST]Support AI/ML Models #233

Open RamachandranSitaraman-IB opened 2 months ago

RamachandranSitaraman-IB commented 2 months ago

Is your feature request related to a problem? Please describe.

No

Describe the solution you'd like

Extend the support in a catalog from Data to AI/ML models. The Polaris catalog could facilitate

  1. Storing metadata about models aiding in model discovery
  2. Integration with MLFlow client
  3. Model provenance
  4. Mapping of Models to the data it operates on
  5. Storing model performance metrics snapshots
  6. Access control on models
  7. Model and its data versioning
  8. Custom properties associated with models
  9. Integration with feature stores
  10. Storage and access of policies associated with model usage
  11. Integration with workflow engines

Describe alternatives you've considered

Have briefly looked at unity catalog, would like to see Polaris supporting AI/ML models too

Additional context

No response