search

helixml / helix

Multi-node production AI stack. Run the best of open source AI easily on your own servers. Create your own AI by fine-tuning open source models. Integrate LLMs with APIs. Run gptscript securely on the server
https://tryhelix.ai
Other
281 stars 18 forks source link

Have a way to not load mistral and sdxl #277

Open eburnette opened 2 months ago

eburnette commented 2 months ago

For self-hosted runners, I just need text inference, so I don't want it to load Mistral for fine-tuning or Sdxl for images.

I tried setting:

RUNTIME_OLLAMA_WARMUP_MODELS=llama3:instruct

but it still loaded Mistral and Sdxl.

Luke suggested one of these options but the runner wouldn't even start:

#RUNTIME_AXOLOTL_WARMUP_MODELS= #RUNTIME_AXOLOTL_WARMUP_MODELS=none

lukemarsden commented 2 months ago

try RUNTIME_AXOLOTL_ENABLED=false

eburnette commented 2 months ago

That causes the runner not to come up. It prints this to its log and nothing else:

2024-05-02 13:08:15 Connecting to controlplane ws://host.docker.internal/api/v1/ws/runner
2024-05-02 13:08:15 2024-05-02T17:08:15Z INF api/cmd/helix/runner.go:321 > Helix runner listening on 0.0.0.0:8080

The dashboard is empty. image