search

lithops-cloud / lithops

A multi-cloud framework for big data analytics and embarrassingly parallel jobs, that provides an universal API for building parallel applications in the cloud ☁️🚀
http://lithops.cloud
Apache License 2.0
315 stars 103 forks source link

OVH S3 (Ceph) needs a region to be specified, otherwise fails #1355

Closed abourramouss closed 3 months ago

abourramouss commented 3 months ago

When trying to use OVH S3 backend, which is based on ceph, i run into the following issue:

2024-05-23 15:30:36,263 [INFO] ceph.py:62 -- Ceph client created - Endpoint: https://s3.gra.perf.cloud.ovh.net/
Traceback (most recent call last):
  File "/home/ayman/.local/bin/lithops", line 33, in <module>
    sys.exit(load_entry_point('lithops', 'console_scripts', 'lithops')())
  File "/home/ayman/.local/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/ayman/.local/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/ayman/.local/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ayman/.local/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ayman/.local/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/ayman/Work/lithops/lithops/scripts/cli.py", line 206, in hello
    fexec = lithops.FunctionExecutor(
  File "/home/ayman/Work/lithops/lithops/executors.py", line 113, in __init__
    self.internal_storage = InternalStorage(storage_config)
  File "/home/ayman/Work/lithops/lithops/storage/storage.py", line 352, in __init__
    self.storage.create_bucket(self.bucket)
  File "/home/ayman/Work/lithops/lithops/storage/storage.py", line 98, in create_bucket
    return self.storage_handler.create_bucket(bucket)
  File "/home/ayman/Work/lithops/lithops/storage/backends/ceph/ceph.py", line 93, in create_bucket
    raise e
  File "/home/ayman/Work/lithops/lithops/storage/backends/ceph/ceph.py", line 86, in create_bucket
    self.s3_client.head_bucket(Bucket=bucket_name)
  File "/home/ayman/.local/lib/python3.10/site-packages/botocore/client.py", line 535, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/ayman/.local/lib/python3.10/site-packages/botocore/client.py", line 983, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (400) when calling the HeadBucket operation: Bad Request

This is solved by passing the region to the storage backend.

abourramouss commented 3 months ago

Solved, closing.