search

sgl-project / sglang

SGLang is a fast serving framework for large language models and vision language models.
https://sgl-project.github.io/
Apache License 2.0
5.95k stars 490 forks source link

[Notice] If someone can not run `examples/usage/llava/srt_llava_next_test.py` and meet the `rpc` error or `connection reset by peer` error. #494

Closed Luodian closed 3 months ago

Luodian commented 5 months ago

I found a workaround to fix it.

By adding a custom port here:

runtime = sgl.Runtime(
    model_path="lmms-lab/llama3-llava-next-8b",
    tokenizer_path="lmms-lab/llama3-llava-next-8b-tokenizer",
    port=8000,
)
github-actions[bot] commented 3 months ago

This issue has been automatically closed due to inactivity. Please feel free to reopen it if needed.