garretwilson
commented
4 years ago I am guessing that S3 did not like the redirect target of
/new-ā.
That turned out to be a correct guess. I changed the program to encode the /new-ā redirect path (resulting in /new-%C4%81), and the above error did not occur; the PUT was successful. In addition the Website-Redirect-Location metadata in the console shows /new-%C4%81.
I did a CloudFront distribution of the S3 website, and it looks like the encoded form correctly redirects to the /new-ā key, so this is very good news.
To summarize the issues in a single sentence: S3 automatically encodes key names but not object redirect key names, but there is no documentation that explains this inconsistency, and moreover the SDK exception contains null instead of any explanation when the redirect path contains an unsupported, unencoded character.
(I can guess why there is a discrepancy between the auto-encoding of the key and the redirect location. Even though the documentation indicates that the object redirect location can contain an S3 object key, it is in reality just the actual content of the HTTP Location header which will be sent back unchanged, so it's not really semantically an object key, in S3 doesn't do anything special with it; the communication happens at the HTTP web server level before it gets to S3 I imagine. In any case, this is undocumented, and what I've written here is many times over anything I could find on the web. It would be nice if AWS could provide us some in-depth, authoritative documentation on this!)
Matthew-Han
yasminetalby
MatiMonaco
There are a various related issues; here are the two main ones. I note issues in bold in the description below.
nullcomponents and are not helpful to identify the problem.I'm using
software.amazon.awssdk:bom:2.13.8. I want to have an S3 objectnew-āwith content, and an objectold-āto serve as a redirct tonew-ā.I do a
PUTforold-ā. From the SDK debug messages, I can see that this got encoded automatically (encodedPath) to/my-bucket/new-%C4%81. I infer from this message that S3 will automatically encode S3 object keys as needed for URIs. If so, that is great! But there is no documentation that I can find that explicitly explains the encoding S3 will perform on object key names. The official object key documentation doesn't really address it. It mentions that some keys "likely need to be URL encoded", but that's not helpful and probably not even totally correct. (URL-encoding uses+for spaces, while URI-encoding, the more appropriate encoding, uses%20for spaces.) And what does "likely" mean? If I pre-encode my key names, will S3 then re-encode them, that is, will it re-encode the%signs that I use for encoding, resulting in duplicate (and erroneous) encoding? There is absolutely no guidance on any of this, and AWS leaves us in the dark with conflicting information.After the
PUTtonew-āsucceeds, I try toPUTa zero-byte objectold-ā, setting aWebsite-Redirect-Location(via the SDK) of/new-ā. (Note that the documentation for object redirects is not clear that redirect destination keys need to be in absolute path form, i.e. starting with a slash, but that is a separate issue.) In this case thePUTfails, but I don't know why. Here is the stack trace.Note that the
nullin the exception is completely unhelpful in telling me what went wrong.I am guessing that S3 did not like the redirect target of
/new-ā. If so, why didn't S3 encode the redirect target S3 key automatically, just like it encodes the S3 key itself? And of course, the encoding procedure for the object redirect target key is not documented either.So there are several problems here, exacerbated by missing documentation, and obscured by exceptions containing
nullinstead of useful information.Perhaps I'm doing something wrong in the code, but if so the SDK is not helping me to know what it is.
I hope this list of problems can help you make the SDK better, and I would really appreciate some improvements. Moreover I'll bet I'm not the only developer who would be very thankful for some updated documentation! Thank you.