Closed rickstaa closed 4 months ago
@yondonfu, is there a specific reason why the binary execution proceeds even when an AIModelConfig is not specified? I encountered this issue while trying to activate flags for containers that don't start warm, as I wanted to pass Optimization flags directly to the AIWorker constructor.
@rickstaa
is there a specific reason why the binary execution proceeds even when an AIModelConfig is not specified?
I think it would sense to require a config file for -aiModels
now because the config file is not only used to warm models, but also for specifying prices and determining which models to advertise during capability discovery. Prior to using the config file for configuring prices and capability discovery, the config file was left as optional because the O would just try to execute any request given a model ID. But now, since models have prices and have to be explicitly advertised, the config file is necessary.
@rickstaa
is there a specific reason why the binary execution proceeds even when an AIModelConfig is not specified?
I think it would sense to require a config file for
-aiModels
now because the config file is not only used to warm models, but also for specifying prices and determining which models to advertise during capability discovery. Prior to using the config file for configuring prices and capability discovery, the config file was left as optional because the O would just try to execute any request given a model ID. But now, since models have prices and have to be explicitly advertised, the config file is necessary.
Ah thanks for the explanation. Makes sense. I will change the behavoir!
What does this pull request do? Explain your changes. (required)
This commit adds a new
OptimizationFlags
field to theaiModels
config so that users can forward optimization environment variables to the ai-worker for more information see https://github.com/livepeer/ai-worker/pull/61.Specific updates (required)
OptimizationFlags
field added toaiModels
config.optimizationFlags
with non warm containers.How did you test each of these updates (required)
SFAST
optimization flag in the models configuration file.Does this pull request close any open issues?
No.
Checklist:
make
runs successfully./test.sh
pass - @rickstaa we still need to fix the tests.