search

iterative / mlem

🐶 A tool to package, serve, and deploy any ML model on any platform. Archived to be resurrected one day🤞
https://mlem.ai
Apache License 2.0
717 stars 44 forks source link

Expose GTO's model version in FastAPI's `interface.json` #665

Open aguschin opened 1 year ago

aguschin commented 1 year ago

Follow-up for #664. Sometimes it's desired to store predictions along with the specific model version that returned those predictions. There are at least two ways to support that in MLEM:

  1. Return model version in prediction - then what's returned will be a json like {"prediction": [0.4, 0.6], "version": 0.1.3}. I've seen some generic ML frameworks doing this IIRC.
  2. Return it in interface.json - we already have MLEM version there, so adding model version looks logical image

Regarding how we get this info into the service. Again, there are two approaches:

  1. Add it at mlem.api.save
  2. Allow to specify it when building server

First seems more reasonable to me. Since this will require some under-the-hood integration with GTO, I'd do this after #664 - which have the same decision to make.

fyi @omesser

aguschin commented 1 year ago

For the record, I'm going to take 2nd option (Return version in interface.json).

And it looks like I need https://github.com/iterative/gto/pull/335 completed first to support this in MLEM.

aguschin commented 1 year ago

UPD: getting the version at mlem.api.save doesn't work since at this moment the commit doesn't exist, not mentioning the right GTO git tag. So it should be read once you build a server.