add modelcars for granite and remove the old mistral deployment

rh-aiservices-bu / parasol-insurance

Source for the "Parasol Insurance" Lab

MIT License

28 stars 26 forks source link

Closed rcarrata closed 5 days ago

rcarrata commented 5 days ago

This PR aims to:

Enable ModelCars into the cluster
Add InferenceService (modelCars) and ServingRuntime based on vLLM within RHOAI for deploy granite in ic-shared-llm ns
Remove the "old" deployment of Mistral LLM using k8s resources