allegroai / clearml-server

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
https://clear.ml/docs
Other
381 stars 131 forks source link

self-hosted: webapp can't start without fileserver #176

Open awoimbee opened 1 year ago

awoimbee commented 1 year ago

As I understand it, the fileserver is not needed if one prefers to use another storage medium (like s3). And since the self-hosted file server is unsecure, I prefer to not use it.

But the webapp requires NGINX_FILESERVER_ADDRESS. If NGINX_FILESERVER_ADDRESS is set to clearml-fileserver but clearml-fileserver does not exist (which is what the helm chart does when fileserver.enabled=false), nginx doesn't start:

2023/01/20 11:27:41 [emerg] 38#38: host not found in upstream "clearml-fileserver" in /etc/nginx/nginx.conf:88
nginx: [emerg] host not found in upstream "clearml-fileserver" in /etc/nginx/nginx.conf:88

The workaround I use is to set NGINX_FILESERVER_ADDRESS to a bogus but existing address (clearml-apiserver).

valeriano-manassero commented 1 year ago

Hi @awoimbee , Fileserver is still needed even when using other storage medium. I know there's plan to make it optional but I need @jkhenning on this to confirm. If this is going to be implemented as optional, I can easily update helm chart too 😃

awoimbee commented 1 year ago

Well, I deployed without a file server, it seems to work 🙈

jkhenning commented 1 year ago

@awoimbee the NGINX_FILESERVER_ADDRESS is basically used to set up the reverse proxy serving the WebApp, so bogus values (as long as you don't use them 🙂) will work 👍