Closed Jimmy-Lu closed 3 months ago
ray version is 2.9.0
adding if __name__ == '__main__':
works for me. But in the end there are some logs:
Processed prompts: 100%|ββββββββββββββββββββββββββββββββββββββββββββββββββββββ| 4/4 [00:00<00:00, 18.26it/s, est. speed input: 100.47 toks/s, output: 292.28 toks/s]
Prompt: 'Hello, my name is', Generated text: " Kees. I am a very passionate and professional photographer.\nI've always been"
Prompt: 'The president of the United States is', Generated text: ' the head of state and the government. A. ιθ―― B.'
Prompt: 'The capital of France is', Generated text: ' a city that is full of history, culture and beauty. It is a city'
Prompt: 'The future of AI is', Generated text: ' bright and it will be a part of our lives in some way or the other'
*** SIGTERM received at time=1719989502 on cpu 71 ***
PC: @ 0x7f74170f2374 (unknown) pthread_cond_wait@@GLIBC_2.3.2
@ 0x7f7416dda090 (unknown) (unknown)
@ ... and at least 1 more frames
[2024-07-03 14:51:42,304 E 17965 10670] logging.cc:440: *** SIGTERM received at time=1719989502 on cpu 71 ***
[2024-07-03 14:51:42,304 E 17965 10670] logging.cc:440: PC: @ 0x7f74170f2374 (unknown) pthread_cond_wait@@GLIBC_2.3.2
[2024-07-03 14:51:42,304 E 17965 10670] logging.cc:440: @ 0x7f7416dda090 (unknown) (unknown)
[2024-07-03 14:51:42,304 E 17965 10670] logging.cc:440: @ ... and at least 1 more frames
INFO 07-03 14:51:46 multiproc_worker_utils.py:123] Killing local vLLM worker processes
[rank0]:[W CudaIPCTypes.cpp:16] Producer process has been terminated before all shared CUDA tensors released. See Note [Sharing CUDA tensors]
Are these ok?
these are some benign erros that @njhill should be working on. It should not affect your inference task I think.
yesοΌno effect. just curious
Your current environment
π Describe the bug
If I just run
python vllm_test.py
as above, I got:And then I run
ray stop
, I gotIf I run
ray start --head --num-gpus 2
fisrt, the ray start correct. And then I runpython vllm_test.py
, I got