search

bentoml / OpenLLM

Run any open-source LLMs, such as Llama, Mistral, as OpenAI compatible API endpoint in the cloud.
https://bentoml.com
Apache License 2.0
10.11k stars 640 forks source link

bug: there is no reply from ai chat on Safari Mac OS but ok on Chrome browser #1101

Open YuriyGavrilov opened 1 month ago

YuriyGavrilov commented 1 month ago

Describe the bug

http://localhost:3000/chat/ could not get reply in safari but ok on google chrome.

To reproduce

openllm % openllm hello Detected Platform: macos Detected Accelerators: None ? Select a model phi3 default Yes ? Select a version phi3:3.8b-ggml-q4 Yes ? Select an action 1. Serve the model locally and get a chat server Installing model dependencies(/Users/yuriygavrilov/.openllm/venv/1815991565470164237)... ....

2024-10-21T22:59:07+0300 [INFO] [entry_service:phi3:1] 127.0.0.1:51338 (scheme=http,method=GET,path=/favicon.ico,type=,length=) (status=404,type=text/plain; charset=utf-8,length=9) 0.765ms (trace=64723d0cc1ebc3f5f6481fc4c8b2e4a4,span=ed6f6ccf4e29d44a,sampled=0,service.name=phi3) 2024-10-21T22:59:07+0300 [INFO] [entry_service:phi3:1] 127.0.0.1:51338 (scheme=http,method=GET,path=/chat.txt,type=,length=) (status=404,type=text/plain; charset=utf-8,length=9) 1.948ms (trace=d20db58e2c10552194311103dcd8216f,span=accea7ce8930a513,sampled=0,service.name=phi3) 2024-10-21T22:59:07+0300 [INFO] [entry_service:phi3:1] 127.0.0.1:51342 (scheme=http,method=GET,path=/favicon-16x16.png,type=,length=) (status=404,type=text/plain; charset=utf-8,length=9) 2.143ms (trace=65928dae5699d42cf7ba4166a8937485,span=9cf8b33bd76afaf2,sampled=0,service.name=phi3) 2024-10-21T22:59:12+0300 [INFO] [entry_service:phi3:1] 127.0.0.1:51342 (scheme=http,method=GET,path=/v1/models,type=application/json,length=) (status=200,type=application/json,length=116) 0.745ms (trace=e8331c38a9ef9a2a7a72b836c282cf05,span=d73c19443f6b8c5d,sampled=0,service.name=phi3) 2024-10-21T22:59:17+0300 [INFO] [entry_service:phi3:1] 127.0.0.1:51342 (scheme=http,method=POST,path=/v1/chat/completions,type=application/json,length=281) (status=200,type=text/event-stream; charset=utf-8,length=) 5173.205ms (trace=d6d8f6c10751d6325a4b48b4c404adb7,span=831025f13911b4e1,sampled=0,service.name=phi3)

Logs

No response

Environment

Mac OS Safari

System information (Optional)

No response