lithops-cloud / lithops

A multi-cloud framework for big data analytics and embarrassingly parallel jobs, that provides an universal API for building parallel applications in the cloud ☁️🚀
http://lithops.cloud
Apache License 2.0
317 stars 105 forks source link

Swift config.py params aren't related to the storage implementation, generate_swift_token needs extra parameters on the authentication payload #1352

Closed abourramouss closed 5 months ago

abourramouss commented 5 months ago

I am currently working with OVHCloud, it uses k8s and swift and i have been trying to use Lithops for it, I've run into some issues.

For the Swift backend implementation the config.py REQ_PARAMS = ('auth_url', 'user_id', 'project_id', 'password', 'region') use the name of the parameter itself. In the storage implementation they are expected to be swift_XXXXX, making lithops throw errors due this inconsistency.

Traceback (most recent call last): File "/home/ayman/.local/bin/lithops", line 33, in <module> sys.exit(load_entry_point('lithops', 'console_scripts', 'lithops')()) File "/home/ayman/.local/lib/python3.10/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) File "/home/ayman/.local/lib/python3.10/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/home/ayman/.local/lib/python3.10/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/ayman/.local/lib/python3.10/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/ayman/.local/lib/python3.10/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/home/ayman/Work/lithops/lithops/scripts/cli.py", line 206, in hello fexec = lithops.FunctionExecutor( File "/home/ayman/Work/lithops/lithops/executors.py", line 113, in __init__ self.internal_storage = InternalStorage(storage_config) File "/home/ayman/Work/lithops/lithops/storage/storage.py", line 343, in __init__ self.storage = Storage(storage_config=storage_config) File "/home/ayman/Work/lithops/lithops/storage/storage.py", line 70, in __init__ raise e File "/home/ayman/Work/lithops/lithops/storage/storage.py", line 66, in __init__ self.storage_handler = StorageBackend(self.config[self.backend]) File "/home/ayman/Work/lithops/lithops/storage/backends/swift/swift.py", line 36, in __init__ self.auth_url = swift_config['swift_auth_url'] KeyError: 'swift_auth_url'

On the other hand, for OVH/swift after fixing the inconsistency, we get the following error: Exception: 401 - UNAUTHORIZED - The request you have made requires authentication.

This issue was resolved by including the domain information in the payload within the generate_swift_token method.