[Bug]: High Level Python SDK throws BadRequestException when referencing missing repository

logan-hcg commented 9 months ago

What happened?

What actually happened, including error codes if applies.

Steps to Reproduce:

import lakefs

lakefs.Repository("<not_real>").branch("branch-name").object("something").exists()

throws:

Traceback (most recent call last):
  File "<path>.venv/lib/python3.12/site-packages/lakefs/exceptions.py", line 139, in api_exception_handler
    yield
  File "<path>.venv/lib/python3.12/site-packages/lakefs/object.py", line 705, in exists
    self._client.sdk_client.objects_api.head_object(self._repo_id, self._ref_id, self._path)
  File "<path>.venv/lib/python3.12/site-packages/pydantic/decorator.py", line 40, in wrapper_function
    return vd.call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path>.venv/lib/python3.12/site-packages/pydantic/decorator.py", line 134, in call
    return self.execute(m)
           ^^^^^^^^^^^^^^^
  File "<path>.venv/lib/python3.12/site-packages/pydantic/decorator.py", line 206, in execute
    return self.raw_function(**d, **var_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path>.venv/lib/python3.12/site-packages/lakefs_sdk/api/objects_api.py", line 922, in head_object
    return self.head_object_with_http_info(repository, ref, path, range, **kwargs)  # noqa: E501
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path>.venv/lib/python3.12/site-packages/pydantic/decorator.py", line 40, in wrapper_function
    return vd.call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path>.venv/lib/python3.12/site-packages/pydantic/decorator.py", line 134, in call
    return self.execute(m)
           ^^^^^^^^^^^^^^^
  File "<path>.venv/lib/python3.12/site-packages/pydantic/decorator.py", line 206, in execute
    return self.raw_function(**d, **var_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path>.venv/lib/python3.12/site-packages/lakefs_sdk/api/objects_api.py", line 1028, in head_object_with_http_info
    return self.api_client.call_api(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path>.venv/lib/python3.12/site-packages/lakefs_sdk/api_client.py", line 407, in call_api
    return self.__call_api(resource_path, method,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path>.venv/lib/python3.12/site-packages/lakefs_sdk/api_client.py", line 222, in __call_api
    raise e
  File "<path>.venv/lib/python3.12/site-packages/lakefs_sdk/api_client.py", line 212, in __call_api
    response_data = self.request(
                    ^^^^^^^^^^^^^
  File "<path>.venv/lib/python3.12/site-packages/lakefs_sdk/api_client.py", line 439, in request
    return self.rest_client.head_request(url,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path>.venv/lib/python3.12/site-packages/lakefs_sdk/rest.py", line 251, in head_request
    return self.request("HEAD", url,
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path>.venv/lib/python3.12/site-packages/lakefs_sdk/rest.py", line 223, in request
    raise BadRequestException(http_resp=r)
lakefs_sdk.exceptions.BadRequestException: (400)
Reason: Bad Request

Expected behavior

I would expect an exception to not be thrown, but instead return False (ie that the object does not exist)

lakeFS version

1.9.1

How lakeFS is installed

Kubernetes

Affected clients

Python lakefs client 0.3.0

Relevant log output

No response

Contact details

No response

yonipeleg33 commented 5 months ago

Hi @logan-hcg, thank you for reporting! You're seeing this exception because the repository name is invalid (contains <>) and not because it does not exist. If you try instead with this code, you'll get False back:

lakefs.Repository("not-real").branch("branch-name").object("something").exists()

However, there's still room for improvement in the error message: If you run the following code:

lakefs.Repository("<invalid-name>").metadata

You get this error message, which contains an informative message in the response body:

lakefs.exceptions.BadRequestException: code: 400, reason: Bad Request, body: {'message': 'argument repository: repository id: invalid value: validation error'}

Lakefs should return the same body when running the code you provided as well.

cc @N-o-Z

logan-hcg commented 5 months ago

@yonipeleg33 that was just a placeholder value, not the real name (which would leak private internal info). The real name did not have <> in it.

However, I will retest, as this may have been fixed since I reported it.

treeverse / lakeFS