Browse Files: Failed to load file browser

[xMinerv]

After installing 0.34.1, the server will run for a couple of days (1 to 2), then when i select the file browser (on the top left) upon log in, I am presented with the error message: Failed to load file browser, and this message in the console:

[150108-10:51] 172.16.xx.xxx - - [08/Jan/2015:10:51:42] "POST /api/listdir HTTP/1.0" 500 805 "https://my.web.address/" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0"
[150108-10:51] ERROR   : [08/Jan/2015:10:51:49] HTTP Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 656, in respond
    response.body = self.handler()
  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 188, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cpdispatch.py", line 34, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/home/myfolder/cherrymusic/cherrymusicserver/httphandler.py", line 290, in api
    return json.dumps({'data': handler(**handler_args)})
  File "/home/myfolder/cherrymusic/cherrymusicserver/httphandler.py", line 452, in api_listdir
    return [entry.to_dict() for entry in self.model.listdir(directory)]
  File "/home/myfolder/cherrymusic/cherrymusicserver/cherrymodel.py", line 126, in listdir
    allfilesindir = [f for f in allfilesindir if not f.startswith('.')]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

if i restart the server, all is working again for a few days.

[devsnd]

Hello xMinerv,

Thanks for your report! Unfortunately, I cannot reproduce the problem on my machine, but I'm using python 3 to run CM, which causes less problems with encodings in general.

So as a quick remedy, you could use python3 instead of python2 to run CM, until we found the underlying issue.

It seems that you have a folder in your basedir with a name that starts with an accented letter (é or so), but for some reason CM reads the file as ASCII...

@tilboerner, IIRC the os.listdir call should return unicode strings, when given a unicode path as parameter, as do all string and path manipulation functions. Can you check if the media.basedir config always returns unicode to begin with?

[tilboerner]

@xMinerv: I found a bug that should have caused the error you describe every time the file browser was loaded when CM was running in Python 2. Did browse files sometimes work for you after the update, or did you only click the button every couple of days?

@devsnd: strings in the config are always unicode strings. Details in the commit above.

Let's make a release once this is fixed. I made a branch prepare-0.34.2 from master and merged it into devel.

[xMinerv]

No, it sometimes worked. I would just come back a few days later, and it was messed up again. a restart of CM allows it to work again for a period of time.

On Sat, Jan 10, 2015 at 11:19 AM, Til Boerner notifications@github.com wrote:

@xMinerv https://github.com/xMinerv: I found a bug that should have caused the error you describe every time the file browser was loaded. Did browse files sometimes work for you after the update, or did you only click the button every couple of days?

@devsnd https://github.com/devsnd: strings in the config are always unicode strings. Details in the commit above.

Let's make a release once this is fixed. I made a branch prepare-0.34.2 https://github.com/devsnd/cherrymusic/tree/prepare-0.34.2 from master and merged it into devel.

— Reply to this email directly or view it on GitHub https://github.com/devsnd/cherrymusic/issues/506#issuecomment-69468138.

[tilboerner]

Ok, I can't reproduce this in a live server, either. @devsnd, you're also right: the only way I can trigger this exception is if I manually force media.basedir to a bytestring. This works by completely bypassing the config api in a test environment; I still can't find a way for it to be anything other than unicode in the server, much less start out as unicode and change types later, as @xMinerv's description suggests.

@xMinerv, could you please do two things to help me narrow this down?

1. Send me (tilman.boerner-at-gmx.net) the following:

   • the output of running cherrymusic --info in python2;
   • your cherrymusic.conf file (see previous output for location);
   • the output of running ls in your music folder (basedir in config), or this if you're on windows.

   Feel free to redact as you see fit, but please preserve any non-ASCII characters.

2. Check out cherrymusic from devel branch and see if the issue still occurs when you run it.

Thanks a lot!

[xMinerv]

I have sent the information requested in step 1.

[devsnd]

@tilboerner Maybe we should make sure that json.loads always uses 'UTF-8' as encoding in https://github.com/devsnd/cherrymusic/blob/devel/cherrymusicserver/httphandler.py#L285 ? Any other hints from data @xMinerv provided?

[tilboerner]

Any other hints from data @xMinerv provided

Nah, everything looks fine.

Maybe we should make sure that json.loads always uses 'UTF-8' as encoding

cherrypy seems to provide GET/POST data values as unicode strings; but I haven't dug into their code yet to see what exactly happens there.

[devsnd]

@xMinerv

I just had a similar problem and it turned out to be the fault of supervisord, so in case you are using supervisor to start cherrymusic, you may want to force the environment encoding of supervisor:

in /etc/supervisor/supervisord.conf

[supervisord]
...
# add this line:
environment=LANG="en_US.utf8", LC_ALL="en_US.UTF-8", LC_LANG="en_US.UTF-8"
...

after that you need to restart the supervisord service. I hope it helps.

[tilboerner]

@devsnd, could you post steps to reproduce? (I.e., is it enough to set a non-unicode locale in the server environment? After that, when does the error occur?) I'd like to snoop around some more and find out what's going on.

[devsnd]

No, the locale on my server is set to en_US.utf-8. But supervisord seems to run python2 and apparently does not correctly use those environment variables.

The server is running Ubuntu 14.04 LTS, so maybe you can reproduce it in a VM. I'll can senf you the config files of my current setup if you want.

Furthermore: maybe we could write some tests that run within a supervisor'ed environment to find out...

[lordkitsuna]

Thanks again, one more question. i got it installed and i have the dependency installed. How can i check to ensure its using opus when available? The song is playing but i dont seem to see an area that will show me with what it decided to transcode the music to.

[devsnd]

right New i only implemented opus decoding, so hat Browsers hat do not support opus are able to play it as another Format (ogg or mp3). to support opus as the target Format i'll have to see if the playback Backend (jplayer) supports it. if they do, it's easy to add support, if not this might be a little trickier.

On March 2, 2015 5:21:17 AM CET, lordkitsuna notifications@github.com wrote:

Thanks again, one more question. i got it installed and i have the dependency installed. How can i check to ensure its using opus when available?

---

Reply to this email directly or view it on GitHub: https://github.com/devsnd/cherrymusic/issues/506#issuecomment-76656825

[6arms1leg]

@lordkitsuna (and @devsnd), I think you are posting to the wrong issue here. You are talking about #517, right? Can we move the conversation back there?