ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
secure key is correctly fetched from the config file.
When self._conf.update is called right after, secure is automatically changed to True.
Testing Instructions
Install latest version of clearml (1.14.3). Try with secure: false in clearml.conf under aws.s3.credentials.
You should get a similar error:
[...]
File "/usr/local/lib/***3.8/site-packages/botocore/retryhandler.py", line 233, in __call__
return self._check_caught_exception(
File "/usr/local/lib/***3.8/site-packages/botocore/retryhandler.py", line 376, in _check_caught_exception
raise caught_exception
File "/usr/local/lib/***3.8/site-packages/botocore/endpoint.py", line 249, in _do_get_response
http_response = self._send(request)
File "/usr/local/lib/***3.8/site-packages/botocore/endpoint.py", line 321, in _send
return self.http_session.send(request)
File "/usr/local/lib/***3.8/site-packages/botocore/httpsession.py", line 466, in send
raise SSLError(endpoint_url=request.url, error=e)
botocore.exceptions.SSLError: SSL validation failed for https://X.X.X.X:9000/bucket/.clearml.bb4a79eb-bdbb-4c34-8e88-6b450ddc9def.test [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1131)
During handling of the above exception, another exception occurred:
[...]
File "/usr/local/lib/***3.8/site-packages/clearml/task.py", line 609, in init
task.output_uri = str(cls.__default_output_uri)
File "/usr/local/lib/***3.8/site-packages/clearml/task.py", line 1207, in output_uri
helper.check_write_permissions(value)
File "/usr/local/lib/***3.8/site-packages/clearml/storage/helper.py", line 2789, in check_write_permissions
raise ValueError("Insufficient permissions (delete failed) for {}".format(base_url))
ValueError: Insufficient permissions (delete failed) for s3://X.X.X.X:9000/bucket/
Related Issue \ discussion
After https://github.com/allegroai/clearml/commit/b78e9b904742073b30558ee06d417770aed0bc4d, secure is now default to
True
. We need to forwardsecure
key otherwise, even if it has been set toFalse
, it switches back toTrue
.Patch Description
At https://github.com/allegroai/clearml/blob/0594a5ac8504d090bb851265f4f5c02c8d862dde/clearml/storage/helper.py#L2069
secure
key is correctly fetched from the config file.When
self._conf.update
is called right after,secure
is automatically changed toTrue
.Testing Instructions
Install latest version of clearml (1.14.3). Try with
secure: false
inclearml.conf
underaws.s3.credentials
. You should get a similar error: