Closed vlaci closed 4 months ago
Thanks for reaching out - this has come up a few times before, I'll share the recent comment that I made here on a related issue to provide more context:
...Related issues have come up here a few times before, for example https://github.com/boto/boto3/issues/2499. Both the HeadObject and HeadBucket APIs return 404 errors if the object or bucket does not exist. And as noted in the documentation:
A
HEAD
request has the same options as aGET
operation on an object. The response is identical to theGET
response except that there is no response body. Because of this, if theHEAD
request generates an error, it returns a generic code, such as400 Bad Request
,403 Forbidden
,404 Not Found
,405 Method Not Allowed
,412 Precondition Failed
, or304 Not Modified
. It's not possible to retrieve the exact exception of these error codes.So this is the behavior of the underlying S3 API and therefore not something that will be addressed in Boto3. As mentioned in previous issues it would be a breaking change to update the model. But please use the Provide feedback at the bottom of API documentation pages to send any additional feedback directly to the S3 team.
I've been digging a bit further and finally figured out why botocore
is unable to handle these cases for HEAD request.
I was initially under the impression that based on the current schema, it is possible for botocore
to "recover" the error code from the HTTP status code given that all HTTP status codes can be mapped unambiguously to error codes (per endpoint). Unfortunately, this is not the case. Although there are only 3(!) HEAD
requests in the whole API and 2 of them are HeadObject
and HeadBucket
which has a single error code specified, but Media Store's DescribeObject
has 2 error cases^1 which result in HTTP 404 status codes (in addition to a bunch of common 400 errors), so my assumption is proven wrong.
Describe the bug
I am not sure if the following issue is an error in documentation or implementation.
The AWS SDK documentation of
HeadObject
^1 states that it signals theNoSuchKey
error with an HTTP 404 status code.botocore
's documentation^2 also mentionsS3.Client.exceptions.NoSuchKey
exception. This exception however is not raised when usinghead_object
on a non-existing object, instead a genericClientError
is received.According to my limited understanding of
botocore
it always parses specific errors from the response body. AsHeadObject
response does not contain a body, it falls back to use the HTTP status code to construct the exception object.Expected Behavior
Preferably,
Head*
andGet*
calls should return the same specific (e.g.NoSuchKey
) exception when called with the same arguments. I understand, that the exception details won't be present forHead*
variants.Alternatively, the documentation of
Head*
methods should be changed, mentioning that only the genericClientError
can be raised in this case.Current Behavior
Client.head_*
calls raise genericClientError
instead of endpoint-specific ones based on status code.Reproduction Steps
Possible Solution
The service description contains the mapping between status code and error code[^3], so
botocore
could be able to figure out the specific exception in this case as well.[^3]: https://github.com/boto/botocore/blob/8b29f58b6f09935e5ec83f4adade36e81643d32f/botocore/data/s3/2006-03-01/service-2.json#L772C19-L772C28 and https://github.com/boto/botocore/blob/8b29f58b6f09935e5ec83f4adade36e81643d32f/botocore/data/s3/2006-03-01/service-2.json#L7476C1-L7476C38
Additional Information/Context
No response
SDK version used
1.34.89
Environment details (OS name and version, etc.)
NixOS 24.05.20240509.6c8fca0 (Uakari) with Python 3.11.9