The test suite assumes the server's environment with 2 available GPUs. As a result, tests will fail on insufficient hardware.
Extend the vllm_modified file to detect what environment we are on and automatically switch to a CPU-based LLM implementation so that tests can still be adequately ran.
Create a separate requirements file for development; as vllm only supports Linux.
To manage separate requirements file, may as well set up the entire repository as a package that can be self-imported (pip install -e .), allowing us to remove these pesky relative imports as well (and run anything from anywhere in the package).
The test suite assumes the server's environment with 2 available GPUs. As a result, tests will fail on insufficient hardware.
vllm_modifiedfile to detect what environment we are on and automatically switch to a CPU-based LLM implementation so that tests can still be adequately ran.vllmonly supports Linux.pip install -e .), allowing us to remove these pesky relative imports as well (and run anything from anywhere in the package).