OOTB support for HF models

opendatahub-io / kserve

Standardized Serverless ML Inference Platform on Kubernetes

https://kserve.github.io/website/

Apache License 2.0

1 stars 23 forks source link

Open heyselbi opened 1 year ago

heyselbi commented 1 year ago

From req doc: 2d P0: Users must be able to serve models from Hugging Face without having to do any additional conversions or configurations

Acceptance criteria:

heyselbi commented 1 year ago

Related to TGIS/Caikit split

heyselbi commented 1 year ago

TGIS image is ready to be tested with HF models.

bdattoma commented 1 year ago

@heyselbi I'll use the ServingRuntime definition from https://github.com/opendatahub-io/caikit-tgis-serving/pull/132/files. Is that correct?

heyselbi commented 1 year ago

@bdattoma that is correct. Let me know if you run into any issues.

bdattoma commented 12 months ago

@bdattoma that is correct. Let me know if you run into any issues.

okay thanks. I'll defer testing this since this ticket got removed from 2.5. I'll be back on it after 2.5