search

allegroai / clearml-server

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
https://clear.ml/docs
Other
364 stars 132 forks source link

Changing the fileserver location to a NAS server-side #212

Closed SalzingerJ closed 11 months ago

SalzingerJ commented 11 months ago

Hi !

We were wondering if it is possible to set the default fileserver location on the side of the tracking server. We have found this ticket https://github.com/allegroai/clearml-server/issues/153 which explains how to do this client-side, but that would force all users and all Agents interacting with the Datasets to have credentials for the NAS. We would prefer to have the default fileserver location of the tracking server to be the NAS (for example mounted locally to the server). That may, we are assuming that the tracking server would the one handling the NAS communication and only it needs credentials.

Is this possible ?

jkhenning commented 11 months ago

Hi @SalzingerJ,

If you have an NFS mounted to /mnt/to/data in your server, you can pass the env var CLEARML_UPLOAD_FOLDER=/mnt/to/data to the fileserver to instruct it to use it as the upload folder. Please note that you'll need to make sure this path will also be mounted into the fileserver docker container.

SalzingerJ commented 11 months ago

Thank you for the answer :) For reference, I was also suggested via the Slack channel this other solution, which I used and it worked perfectly : I can simply mount the NAS folder to /opt/clearml/data/fileserver and it will work out of the box.