search

containers / ai-lab-recipes

Examples for building and running LLM services and applications locally with Podman
Apache License 2.0
103 stars 106 forks source link

Fix env passing that leads to failures #731

Closed n1hility closed 1 month ago

n1hility commented 1 month ago

vLLM fails with empty set values. Adjust the model of env passing to only set a value if it is defined.

Also drops ILAB_GLOBAL_CONFIG which needs to be set by the container

romfreiman commented 1 month ago

u have to update https://github.com/containers/ai-lab-recipes/blob/main/training/nvidia-bootc/duplicated/ilab-wrapper/ilab as well...

n1hility commented 1 month ago

ah yes forgot we did that

eranco74 commented 1 month ago

This fix the ValueError: Unable to configure handler 'vllm' error

cdvultur commented 1 month ago

Double checked it and it is working fine on my setup, by applying the changes from this PR to existing wrapper.