If you just wanted to deploy RHEL AI as an inference server using Granite as the LLM and vLLM as the API server, what is the correct procedure? Is this easy to do? Is there already a boot image or container image for this? If it can be done, would you be prepared to document it for those of us needing a quickstart guide?
Hi There!
If you just wanted to deploy RHEL AI as an inference server using Granite as the LLM and vLLM as the API server, what is the correct procedure? Is this easy to do? Is there already a boot image or container image for this? If it can be done, would you be prepared to document it for those of us needing a quickstart guide?