PyFilesystem / pyfilesystem

Python filesystem abstraction layer
http://pyfilesystem.org/
BSD 3-Clause "New" or "Revised" License
288 stars 63 forks source link

S3FS doesn't work on python3, encoding problem. #188

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
What steps will reproduce the problem?
1. Start using python 3 (in my case 3.4.2)
2. from fs.s3fs import S3FS
3. _s3fs = S3FS(...)
4. _s3fs.listdir('')
Listdir, or pretty much any other function call, where you need to pass a 
string will return an error.

What is the expected output? What do you see instead?
expected output would be the dir I wanted to check listed, the real output is:

/Users/_____/Workspace/virtualenvs/_____/lib/python3.4/site-packages/fs/s3fs.py 
in _s3path(self, path)
    178         path = relpath(normpath(path))
    179         path = self._separator.join(iteratepath(path))
--> 180         s3path = self._prefix + path
    181         if s3path and s3path[-1] == self._separator:
    182             s3path = s3path[:-1]

TypeError: can't concat bytes to str

What version of the product are you using? On what operating system?
OSX 10.10 Yosemite, fs 0.5.0, python 3.4.2, Boto 2.33.0

Please provide any additional information below.
The problem is: when you pass the prefix on the initialization of the S3FS it 
decodes it using utf8, so it becomes a bytes variable, instead of a string. 
Then when you pass the argument to any of the functions, it tries to 
concatenate it to the self._prefix, but as you must pass a string and not a 
bytes argument, it gives you this error.
You can't pass a bytes argument because of the regex that checks if 
normalization is required, if a bytes is sent, it will return an error saying: 
TypeError: can't use a string pattern on a bytes-like object

If the argument passed to the function was to be decoded before you concatenate 
it to the prefix it should work, or changing the regex to something else.

Original issue reported on code.google.com by lvie...@lvieira.com on 17 Oct 2014 at 9:56

KennethAdamMiller commented 8 years ago

I'm having the same problem.