Serverless Mixtral Runpod

TrelisResearch / one-click-llms

One click templates for inferencing Language Models

104 stars 12 forks source link

Closed anujnayyar1 closed 6 months ago

anujnayyar1 commented 6 months ago

Any chance you could turn this template into a serverless one akin to: https://github.com/runpod-workers/worker-tgi

RonanKMcGovern commented 6 months ago

Yeah, thanks for the suggestion, it's on my list of vids. Let me target that soon.

RonanKMcGovern commented 6 months ago

Howdy, so I'm in touch with Runpod about supporting serverless templates again.

In the meantime, I've just released this explainer video.