Open youkaichao opened 3 months ago
hmm, these should only be called when usin ADAG. Were these environment variables set? DISTRIBUTED_EXECUTOR_BACKEND=ray VLLM_USE_RAY_SPMD_WORKER=1 VLLM_USE_RAY_COMPILED_DAG=1 @youkaichao
Also to be clear, we are planning to fix this soon (and it is supposed to happen only with env var like @ruisearch42 said above). If it happens without an env var, we will try fixing it immediately, otherwise it will take a few more days until we tackle it.
it happens when i use these env vars. so it is not user-facing now.
I just came across it when I use ray dag for testing. fixing it later should be fine.
This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!
Your current environment
🐛 Describe the bug
command:
python benchmarks/benchmark_throughput.py --input-len 100 --output-len 100 --num-prompts 100 --model facebook/opt-125m -tp 2 --distributed-executor-backend ray
error:
cc @ruisearch42 @rkooo567 @stephanie-wang