search

xach / zs3

Work with Amazon S3 and Amazon CloudFront from Common Lisp
http://www.xach.com/lisp/zs3/
Other
38 stars 16 forks source link

Slashes in keys are urlencoded when creating signed URL #12

Closed hanshuebner closed 8 years ago

hanshuebner commented 10 years ago

Basically, everything is described here: http://stackoverflow.com/questions/15473582/amazon-s3-presigned-urls-escape-the-slashes-in-the-key - ZS3, like some common Java libraries, urlencodes the key (and the bucket, sub-resource) when generating a signed path, which basically results all path components to be folded into the pathname of the resulting URL. That causes the issues when particular file names are desired for files downloaded from S3.

Do you have any thoughts regarding this? I'm inclined to remove the url-encode invocations from signed-path and authorized-url, but maybe you have a specific reason why all components are URL encoded.

xach commented 10 years ago

My initial thought, without reading the code or commit history, is that it had something to do with PURI's escape policies.

I'll try to check it out soon.

hanshuebner commented 10 years ago

I have removed the url-encode's in our own fork to see whether that affects production for us, and it does not. It certainly restricts the character set allowable in keys and bucket names, but that is acceptable to us. Maybe an option would make sense?

2014-10-03 8:21 GMT-04:00 Zach Beane notifications@github.com:

My initial thought, without reading the code or commit history, is that it had something to do with PURI's escape policies.

I'll try to check it out soon.

Reply to this email directly or view it on GitHub https://github.com/xach/zs3/issues/12#issuecomment-57788326.

xach commented 9 years ago

Here are the tests that fail for me:

Both of these seem like reasonable key names to me, so I can't just stop the URL encoding entirely. I'll look into whether I can sign the url-encoded version but return something with at least slashes un-encoded.

xach commented 9 years ago

So you have a fork that doesn't url-encode things at all? Do you use keys with "/" in them? If so, can I see a patch to try it out locally? When I remove url-encoding, I get signature failures.

hanshuebner commented 9 years ago

We are using https://github.com/mcna/zs3 in production and it works with keys that contain slashes. It does not work, however, with keys that contain spaces - Not a requirement on our end. I guess that a specialized url-encode function would make the most sense.

xach commented 8 years ago

Happy coincidence: This will be fixed as part of the AWS4 auth updates.

xach commented 8 years ago

Fixed in 1.2.8.