Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
We encountered a bug when serving Xinference (since 0.8.5 & through docker) through an apache vhost with reverse proxy configuration. Issue is the dynamic resolution of paths (through gradio>=4.0.0) in the chat ui. URLs pointing to http://$HOST/$MODEL_ID/info and http://$HOST/$MODEL_ID/theme.css will be instead resolved as http://$HOST/info and http://$HOST/theme.css. This makes the chat ui unusable when served through an apache vhost.
The issue was introduced with commit #983, removing the hard requirement of gradio<4.0.0.
To Reproduce
Launch latest (0.10.0) Xinference docker image.
Install apache vhost and configure reverse proxy:
<VirtualHost *:80>
ServerName $server_name
#default proxy via internal port
ProxyPreserveHost On
RewriteEngine on
RewriteCond %{HTTP:Upgrade} =websocket [NC]
RewriteCond %{REQUEST_URI} ^/
RewriteRule /(.*) ws://localhost:9997/$1 [P,L]
RewriteRule ^/(.*)$ http://127.0.0.1:9997/$1 [P]
Header always set Content-Security-Policy: upgrade-insecure-requests
HostnameLookups Off
UseCanonicalName Off
ServerSignature On
<Directory />
AllowOverride None
Options -Indexes
</Directory>
TraceEnable off
Errorlog /usr/share/apache2/log/llm/llm-error.log
CustomLog /usr/share/apache2/log/llm/llm-access.log combined
3. Launch a model in the UI
4. Open chat UI
5.
![image](https://github.com/xorbitsai/inference/assets/18685571/2445ce2f-d32d-4b0f-af9d-c3306394f02b)
### To Fix
I created a derived version of the official Xinference docker image as follows:
```Dockerfile
FROM xprobe/xinference:latest
RUN pip install gradio==3.50.2
Repeating step 1.-4. for this docker image will result in a working chat UI.
Therefore I propose to fix the gradio dependency to 3.50.2.
Describe the bug
We encountered a bug when serving Xinference (since 0.8.5 & through docker) through an apache vhost with reverse proxy configuration. Issue is the dynamic resolution of paths (through gradio>=4.0.0) in the chat ui. URLs pointing to http://$HOST/$MODEL_ID/info and http://$HOST/$MODEL_ID/theme.css will be instead resolved as http://$HOST/info and http://$HOST/theme.css. This makes the chat ui unusable when served through an apache vhost.
The issue was introduced with commit #983, removing the hard requirement of gradio<4.0.0.
To Reproduce
Install apache vhost and configure reverse proxy:
Repeating step 1.-4. for this docker image will result in a working chat UI.
Therefore I propose to fix the gradio dependency to 3.50.2.