xorbitsai / inference

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
https://inference.readthedocs.io
Apache License 2.0
3.57k stars 296 forks source link

BUG: Issue with chat ui because of gradio when using apache vhost as reverse proxy #1282

Open AndiMajore opened 2 months ago

AndiMajore commented 2 months ago

Describe the bug

We encountered a bug when serving Xinference (since 0.8.5 & through docker) through an apache vhost with reverse proxy configuration. Issue is the dynamic resolution of paths (through gradio>=4.0.0) in the chat ui. URLs pointing to http://$HOST/$MODEL_ID/info and http://$HOST/$MODEL_ID/theme.css will be instead resolved as http://$HOST/info and http://$HOST/theme.css. This makes the chat ui unusable when served through an apache vhost.

The issue was introduced with commit #983, removing the hard requirement of gradio<4.0.0.

To Reproduce

  1. Launch latest (0.10.0) Xinference docker image.
  2. Install apache vhost and configure reverse proxy:

    
    <VirtualHost *:80>
     ServerName $server_name
    
     #default proxy via internal port
     ProxyPreserveHost On
    
     RewriteEngine on
     RewriteCond %{HTTP:Upgrade} =websocket [NC]
     RewriteCond %{REQUEST_URI} ^/
     RewriteRule /(.*) ws://localhost:9997/$1 [P,L]
    
     RewriteRule ^/(.*)$ http://127.0.0.1:9997/$1 [P]
    
     Header always set Content-Security-Policy: upgrade-insecure-requests
    
     HostnameLookups Off
    
     UseCanonicalName Off
    
     ServerSignature On
    
    <Directory />
     AllowOverride None
     Options -Indexes
    </Directory>
     TraceEnable off
    
     Errorlog /usr/share/apache2/log/llm/llm-error.log
     CustomLog /usr/share/apache2/log/llm/llm-access.log combined

3. Launch a model in the UI
4. Open chat UI
5. 
![image](https://github.com/xorbitsai/inference/assets/18685571/2445ce2f-d32d-4b0f-af9d-c3306394f02b)

### To Fix
I created a derived version of the official Xinference docker image as follows:

```Dockerfile
FROM xprobe/xinference:latest
RUN pip install gradio==3.50.2

Repeating step 1.-4. for this docker image will result in a working chat UI.

image

Therefore I propose to fix the gradio dependency to 3.50.2.

qinxuye commented 2 months ago

@ChengjieLi28 Do you think it's reasonable to fix the gradio version for docker build?