Trying to load models from behind a proxy is not possible

[robinvd]

Describe the bug

Trying to load models from behind a proxy is not possible

To reproduce

We self host clearml, and its not generally available. Because of a little weird proxy setup its not always available at the same url. But with settings the correct env variables it all works fine. Except when trying to load files. The metadata is loaded from the api but then it will just blindly download the url specified in the artifact/model.

example model:

ID: fa09d469325c4281845381db82ddbac0
CREATED AT: Jun 7 2023 11:32
UPDATED AT: Jun 7 2023 11:32
FRAMEWORK: ONNX
STATUS: Draft
MODEL URL: https://files.main-clearml-url/trainings/train.f18fd5ac4d3b48cfb1bb1ccfeb937506/models/model.onnx

# run.py
InputModel("fa09d469325c4281845381db82ddbac0").get_local_copy()

export CLEARML_WEB_HOST=https://web.proxy-to-clearml
export CLEARML_API_HOST=https://api.proxy-to-clearml
export CLEARML_FILES_HOST=https://files.proxy-to-clearml
./run.py

output (from clearml-serving)

2023-07-04 11:40:28,479 - clearml.storage - ERROR - Could not download https://files.main-clearml-url/trainings/train.f18fd5ac4d3b48cfb1bb1ccfeb937506/models/model.onnx , err: Failed getting object https://files.main-clearml-url/trainings/train.f18fd5ac4d3b48cfb1bb1ccfeb937506/models/model.onnx (403): Forbidden
Error retrieving model ID fa09d469325c4281845381db82ddbac0 []

Expected behaviour

The model is loaded correctly

Environment

• Server type: self hosted
• ClearML SDK Version
• ClearML Server Version (Only for self hosted). WebApp: 1.10.1-359 • Server: 1.10.1-359 • API: 2.24
• Python Version
• OS: Linux

  Related Discussion

  If this continues a slack thread, please provide a link to the original slack thread.