DocMarty84 / koozic

Self-hosted media streaming server
https://koozic.net
Other
90 stars 12 forks source link

Memory /CPU usage #7

Closed aroundmyroom closed 6 years ago

aroundmyroom commented 6 years ago

I see that python /opt/koozic/odoo-bin is rather using a lot of CPU where I think: you do nothing. ;_) also with my large db and running in the VM that builing up a page takes some time

I'll keep watching the development as right now next to subsonic its the best alternative, but I do miss the feeling that it is smooth and slack when pressing songs etc.

DocMarty84 commented 6 years ago

Hum, how "slow" is your installation in comparison with the demo server? It is seems much slower, there might be a few things to improve.

First, what are the parameters assigned to the VM? I would recommand at least 2 CPUs to take advantage of the parallelization of the requests. Note that even if you assigned only 1 CPU, you might give a shot to this:

This will enable the use of 2 processes (workers). You might as well try 4 since Koozic is the "only" service running on your VM.

Moreover, there are a couple of background jobs that might be running. These might kill the performances if only 1 CPU/worker is used. Indeed, your collection being huge, they take some time to run and some of them are resource hungry:

You might want to run these at night, or completely deactivate them.

In the list, some of them start with "oomusic...". You can set a "Next Execution Date" at night (if your VM runs 24/7), or change the frequency (once a week/month instead of once a day), or even deactivate them (untick the "Active" field).

The drawback of deactivation is that what was not computed by these scheduled actions might need to be computed on-the-fly when necessary. For example, if you open the details of an artist and the corresponding LastFM information have never been fetched before, it might take a few seconds before the page opens (just the time to fetch the data). Once the data is fetched, that's kept in cache for some time. So, it's only the first access which is slow. Further accesses won't be.

You can also go to Settings > Parameters > System Parameters, and change the value of "oomusic.lastfm_cache_duration" to 90 or more. LastFM info will be updated every 90 days (or more) instead of 14.

That would be nice if you could give a shot to this. It you can witness a noticeable improvement, I will adapt the default set-up to fit with a huge database.

Finally, I agree that the interface is not super-smooth, even with a perfectly configured system. That's inherent to the development framework used (Odoo).

aroundmyroom commented 6 years ago

Ok .. I will give this a shot .. will update this part during the test when I am able to. Because .. some figures

wonder how long this will take here to be able to modify it;) with 4TB of music for sure this could be the issue causing the slow build-up of the webpage

did a short check yesterday about 'loads' and installed Serviio . the mediabrowser part is working quick so in a sence I think the framework should be optimized and if Odoo is not, it will be difficult ;)

DocMarty84 commented 6 years ago

not yet possible: Record cannot be modified right now: This cron task is currently being executed and may not be modified Please try again in a few minutes

For all of them? I would say it should be the case for 'oomusic.build.lastfm.cache', 'oomusic.build.artists.image.cache' and maybe 'oomusic.build.image.cache'. The others, it should not be the case.

But there is obviously something wrong :-) And I'm like 90 % sure that it comes with the LastFM cache being built. KooZic is going through all your artist list (~8000 I think), and retrieves the information for all of them. That takes time, and potentially acts like a bottleneck for the other requests. If you type journalctl -fe, you might see a lot of messages like 'Connection to https://ws.audioscrobbler.com/....'.

Good thing is that it should be quite easy to avoid this. We can force the deactivation of the scheduled actions quite easily:

Hopefully now you'll see a difference.

28779 pages with artists

That's weird, Subsonic was giving ~8000. Note that KooZic is case sensitive: "the beatles", "The beatles" or "The Beatles" will be recognized as 3 different artists (and therefore fetch the data 3 times). This might explain why it takes so much time for you. For each artist, 3 requests are made and therefore the LastFM cache is running for hours and hours to fetch so many data...

the mediabrowser part is working quick so in a sence I think the framework should be optimized and if Odoo is not, it will be difficult ;)

You can check the demo server, I consider the speed "slightly below average". Note that it runs on a cheap VPS (CPU is 2 x Intel Atom C2750 @ 2.40GHz, 2GB of RAM). The DB is small on the demo server, but that shouldn't really matter. Only the data which is displayed is fetched. So it doesn't matter if you have 10 or 10,000 albums: you'll only display and fetch 40 at a time.

DocMarty84 commented 6 years ago

By the way, Odoo is an ERP software that can handle millions of accounting entries or stock movement ;-) So that's why I'm pretty convinced it can handle a large music collection. I think the major problem you are facing is with this LastFM cache mechanism, which is clearly my mistake.

aroundmyroom commented 6 years ago

your demo is working as expected I see that 1GB of memory can give an issue

also this after restart d=image_small&id=18225&unique=20180418135624 HTTP/1.1" 200 - Apr 21 15:19:54 koozic odoo-bin[1660]: 2018-04-21 13:19:54,765 1675 ERROR koozic odoo.service.server: Worker (1675) Exception occured, exiting... Apr 21 15:19:54 koozic odoo-bin[1660]: Traceback (most recent call last): Apr 21 15:19:54 koozic odoo-bin[1660]: File "/opt/koozic/odoo/service/server.py", line 734, in run Apr 21 15:19:54 koozic odoo-bin[1660]: self.process_work() Apr 21 15:19:54 koozic odoo-bin[1660]: File "/opt/koozic/odoo/service/server.py", line 767, in process_work Apr 21 15:19:54 koozic odoo-bin[1660]: self.process_request(client, addr) Apr 21 15:19:54 koozic odoo-bin[1660]: File "/opt/koozic/odoo/service/server.py", line 758, in process_request Apr 21 15:19:54 koozic odoo-bin[1660]: self.server.process_request(client, addr) Apr 21 15:19:54 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 318, in process_request Apr 21 15:19:54 koozic odoo-bin[1660]: self.finish_request(request, client_address) Apr 21 15:19:54 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 331, in finish_request Apr 21 15:19:54 koozic odoo-bin[1660]: self.RequestHandlerClass(request, client_address, self) Apr 21 15:19:54 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 654, in init Apr 21 15:19:54 koozic odoo-bin[1660]: self.finish() Apr 21 15:19:54 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 713, in finish Apr 21 15:19:54 koozic odoo-bin[1660]: self.wfile.close() Apr 21 15:19:54 koozic odoo-bin[1660]: File "/usr/lib/python2.7/socket.py", line 283, in close Apr 21 15:19:54 koozic odoo-bin[1660]: self.flush() Apr 21 15:19:54 koozic odoo-bin[1660]: File "/usr/lib/python2.7/socket.py", line 307, in flush Apr 21 15:19:54 koozic odoo-bin[1660]: self._sock.sendall(view[write_offset:write_offset+buffer_size]) Apr 21 15:19:54 koozic odoo-bin[1660]: timeout: timed out

and Apr 21 14:47:07 koozic kernel: Out of memory: Kill process 1438 (python) score 675 or sacrifice child Apr 21 14:47:07 koozic kernel: Killed process 1438 (python) total-vm:1696448kB, anon-rss:787068kB, file-rss:1076kB, shmem-rss:0kB Apr 21 14:47:09 koozic odoo-bin[529]: 2018-04-21 12:47:09,530 1538 INFO koozic odoo.service.server: Worker WorkerCron (1538) alive Apr 21 14:48:24 koozic odoo-bin[529]: 2018-04-21 12:48:22,976 1538 INFO koozic odoo.addons.base.ir.ir_cron: Starting job oomusic.build.image.cache. Apr 21 14:55:24 koozic odoo-bin[529]: 2018-04-21 12:55:24,624 529 ERROR koozic odoo.service.server: WorkerCron (1415) timeout after 3600s Apr 21 14:55:24 koozic odoo-bin[529]: 2018-04-21 12:55:24,698 529 ERROR koozic odoo.service.server: WorkerCron (1415) timeout after 3600s Apr 21 14:55:25 koozic odoo-bin[529]: 2018-04-21 12:55:25,015 1558 INFO koozic odoo.service.server: Worker WorkerCron (1558) alive Apr 21 14:56:33 koozic odoo-bin[529]: 2018-04-21 12:56:33,190 1558 INFO koozic odoo.addons.base.ir.ir_cron: Starting job oomusic.build.lastfm.cache.

and accessing the albums page resulted in this

Apr 21 15:28:14 koozic odoo-bin[1660]: 2018-04-21 13:28:14,371 1676 INFO koozic werkzeug: 10.1.1.101 - - [21/Apr/2018 13:28:14] "GET /oomusic/trans/45918.opus?raw=0&seek=0&norm=0 HTTP/1.1" 200 - Apr 21 15:28:32 koozic odoo-bin[1660]: 2018-04-21 13:28:32,043 1676 ERROR koozic odoo.service.server: Worker (1676) Exception occured, exiting... Apr 21 15:28:32 koozic odoo-bin[1660]: Traceback (most recent call last): Apr 21 15:28:32 koozic odoo-bin[1660]: File "/opt/koozic/odoo/service/server.py", line 734, in run Apr 21 15:28:32 koozic odoo-bin[1660]: self.process_work() Apr 21 15:28:32 koozic odoo-bin[1660]: File "/opt/koozic/odoo/service/server.py", line 767, in process_work Apr 21 15:28:32 koozic odoo-bin[1660]: self.process_request(client, addr) Apr 21 15:28:32 koozic odoo-bin[1660]: File "/opt/koozic/odoo/service/server.py", line 758, in process_request Apr 21 15:28:32 koozic odoo-bin[1660]: self.server.process_request(client, addr) Apr 21 15:28:32 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 318, in process_request Apr 21 15:28:32 koozic odoo-bin[1660]: self.finish_request(request, client_address) Apr 21 15:28:32 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 331, in finish_request Apr 21 15:28:32 koozic odoo-bin[1660]: self.RequestHandlerClass(request, client_address, self) Apr 21 15:28:32 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 654, in init Apr 21 15:28:32 koozic odoo-bin[1660]: self.finish() Apr 21 15:28:32 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 713, in finish Apr 21 15:28:32 koozic odoo-bin[1660]: self.wfile.close() Apr 21 15:28:32 koozic odoo-bin[1660]: File "/usr/lib/python2.7/socket.py", line 283, in close Apr 21 15:28:32 koozic odoo-bin[1660]: self.flush() Apr 21 15:28:32 koozic odoo-bin[1660]: File "/usr/lib/python2.7/socket.py", line 307, in flush Apr 21 15:28:32 koozic odoo-bin[1660]: self._sock.sendall(view[write_offset:write_offset+buffer_size]) Apr 21 15:28:32 koozic odoo-bin[1660]: timeout: timed out Apr 21 15:28:32 koozic odoo-bin[1660]: 2018-04-21 13:28:32,352 1720 INFO koozic odoo.service.server: Worker WorkerHTTP (1720) alive Apr 21 15:28:57 koozic odoo-bin[1660]: 2018-04-21 13:28:57,429 1679 INFO koozic odoo.addons.base.ir.ir_cron: Starting job oovideo.scan.folder. Apr 21 15:28:57 koozic odoo-bin[1660]: 2018-04-21 13:28:57,588 1679 INFO koozic odoo.modules.loading: loading 1 modules... Apr 21 15:28:57 koozic odoo-bin[1660]: 2018-04-21 13:28:57,630 1679 INFO koozic odoo.modules.loading: 1 modules loaded in 0.04s, 0 queries Apr 21 15:28:57 koozic odoo-bin[1660]: 2018-04-21 13:28:57,668 1679 INFO koozic odoo.modules.loading: loading 10 modules... Apr 21 15:28:57 koozic odoo-bin[1660]: 2018-04-21 13:28:57,690 1679 INFO koozic odoo.modules.loading: 10 modules loaded in 0.02s, 0 queries Apr 21 15:28:57 koozic odoo-bin[1660]: 2018-04-21 13:28:57,778 1679 INFO koozic odoo.modules.loading: Modules loaded. Apr 21 15:28:57 koozic odoo-bin[1660]: 2018-04-21 13:28:57,873 1679 ERROR koozic odoo.addons.oovideo.models.oovideo_folder_scan: "/media/music" is locked! It probably means that a scan is ongoing.

took around 2 to 3 minutes to have the music playing again and showing this menu.

DocMarty84 commented 6 years ago

Indeed, it seems you are hitting some memory limit. Have you tried to force the deactivation of the scheduled actions (as explained above)?

My assumption is that the cron jobs are using all the memory of your VM, resulting in a very slow interface. RAM was never an issue where I installed KooZic, but here is the example that I need to be more careful about cache flushing.

DocMarty84 commented 6 years ago

By the way, this is something I already improved for v1.0.0 (for the record, I flush the cache every 50 folders). So the memory should not increase in an unexpected way. I might have to double check this everywhere.

Anyway, thanks a lot for your time, it really helps me to have this information.

aroundmyroom commented 6 years ago

yep I did and actually now after some hours I see that the system is running 'smooth' pressing menu items on the left, so even after turning it off and restarting it took some time that all went ok.

now I will check if that will keep running smooth

saw a few things, just after deleting a playlist music was playing, suddenly playing stops during playback and starts at beginning again.

t/read HTTP/1.1" 200 - Apr 21 20:30:30 koozic odoo-bin[1660]: 2018-04-21 18:30:30,611 1720 ERROR koozic odoo.service.server: Worker (1720) Exception occured, exiting... Apr 21 20:30:30 koozic odoo-bin[1660]: Traceback (most recent call last): Apr 21 20:30:30 koozic odoo-bin[1660]: File "/opt/koozic/odoo/service/server.py", line 734, in run Apr 21 20:30:30 koozic odoo-bin[1660]: self.process_work() Apr 21 20:30:30 koozic odoo-bin[1660]: File "/opt/koozic/odoo/service/server.py", line 767, in process_work Apr 21 20:30:30 koozic odoo-bin[1660]: self.process_request(client, addr) Apr 21 20:30:30 koozic odoo-bin[1660]: File "/opt/koozic/odoo/service/server.py", line 758, in process_request Apr 21 20:30:30 koozic odoo-bin[1660]: self.server.process_request(client, addr) Apr 21 20:30:30 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 318, in process_request Apr 21 20:30:30 koozic odoo-bin[1660]: self.finish_request(request, client_address) Apr 21 20:30:30 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 331, in finish_request Apr 21 20:30:30 koozic odoo-bin[1660]: self.RequestHandlerClass(request, client_address, self) Apr 21 20:30:30 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 654, in init Apr 21 20:30:30 koozic odoo-bin[1660]: self.finish() Apr 21 20:30:30 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 713, in finish Apr 21 20:30:30 koozic odoo-bin[1660]: self.wfile.close() Apr 21 20:30:30 koozic odoo-bin[1660]: File "/usr/lib/python2.7/socket.py", line 283, in close Apr 21 20:30:30 koozic odoo-bin[1660]: self.flush() Apr 21 20:30:30 koozic odoo-bin[1660]: File "/usr/lib/python2.7/socket.py", line 307, in flush Apr 21 20:30:30 koozic odoo-bin[1660]: self._sock.sendall(view[write_offset:write_offset+buffer_size]) Apr 21 20:30:30 koozic odoo-bin[1660]: timeout: timed out Apr 21 20:30:31 koozic odoo-bin[1660]: 2018-04-21 18:30:31,429 2353 INFO koozic odoo.service.server: Worker WorkerHTTP (2353) alive Apr 21 20:30:32 koozic odoo-bin[1660]: 2018-04-21 18:30:32,691 1695 INFO koozic werkzeug: 10.1.1.101 - - [21/Apr/2018 18:30:32] "POST /web/action/load HTTP/1.1" 200 - Apr 21 20:30:32 koozic odoo-bin[1660]: 2018-04-21 18:30:32,898 2353 INFO ? odoo.http: HTTP Configuring static files Apr 21 20:30:32 koozic odoo-bin[1660]: 2018-04-21 18:30:32,911 2353 INFO koozic odoo.addons.base.ir.ir_http: Generating routing map Apr 21 20:30:33 koozic odoo-bin[1660]: 2018-04-21 18:30:33,127 2353 INFO koozic werkzeug: 10.1.1.101 - - [21/Apr/2018 18:30:33] "POST /web/dataset/call_kw/oomusic.playlist.line/load_views HTTP/1.1" 200 -

aroundmyroom commented 6 years ago

just turning on the image part . .just to see what is happening

Apr 21 20:42:06 koozic odoo-bin[1660]: 2018-04-21 18:42:06,840 1679 INFO koozic odoo.modules.loading: Modules loaded. Apr 21 20:43:30 koozic odoo-bin[1660]: 2018-04-21 18:43:29,709 1679 INFO koozic odoo.sql_db: bad query: UPDATE "oomusic_folder" SET "write_uid"=1,"write_date"=(now() at time zone 'UTC') WHERE id IN (6974) Apr 21 20:43:30 koozic odoo-bin[1660]: 2018-04-21 18:43:29,766 1679 ERROR koozic odoo.addons.base.ir.ir_cron: Call of self.env[u'oomusic.folder'].cron_build_image_cache(()) failed in Job #5 Apr 21 20:43:30 koozic odoo-bin[1660]: Traceback (most recent call last): Apr 21 20:43:30 koozic odoo-bin[1660]: File "/opt/koozic/odoo/addons/base/ir/ir_cron.py", line 126, in _callback Apr 21 20:43:30 koozic odoo-bin[1660]: getattr(model, method_name)(args) Apr 21 20:43:30 koozic odoo-bin[1660]: File "/opt/koozic/addons/oomusic/models/oomusic_folder.py", line 284, in cron_build_image_cache Apr 21 20:43:30 koozic odoo-bin[1660]: folder._compute_image_big() Apr 21 20:43:30 koozic odoo-bin[1660]: File "/opt/koozic/addons/oomusic/models/oomusic_folder.py", line 144, in _compute_image_big Apr 21 20:43:30 koozic odoo-bin[1660]: 'image_big_cache': resized_images['image'], Apr 21 20:43:30 koozic odoo-bin[1660]: File "/opt/koozic/addons/oomusic/models/oomusic_folder.py", line 231, in write Apr 21 20:43:30 koozic odoo-bin[1660]: return super(MusicFolder, self).write(vals) Apr 21 20:43:30 koozic odoo-bin[1660]: File "/opt/koozic/odoo/models.py", line 3560, in write Apr 21 20:43:30 koozic odoo-bin[1660]: self._write(old_vals) Apr 21 20:43:30 koozic odoo-bin[1660]: File "/opt/koozic/odoo/models.py", line 3646, in _write Apr 21 20:43:30 koozic odoo-bin[1660]: cr.execute(query, params + (sub_ids,)) Apr 21 20:43:30 koozic odoo-bin[1660]: File "/opt/koozic/odoo/sql_db.py", line 154, in wrapper Apr 21 20:43:30 koozic odoo-bin[1660]: return f(self, *args, **kwargs) Apr 21 20:43:30 koozic odoo-bin[1660]: File "/opt/koozic/odoo/sql_db.py", line 231, in execute Apr 21 20:43:30 koozic odoo-bin[1660]: res = self._obj.execute(query, params) Apr 21 20:43:30 koozic odoo-bin[1660]: TransactionRollbackError: could not serialize access due to concurrent update

music playing, suddenly stops and next song in playlist is started

DocMarty84 commented 6 years ago

just turning on the image part . .just to see what is happening

Hum, it seems that the image caching mechanism I added to avoid sluggishness is causing quite a lot of troubles for you... I probably never noticed it since on my side, this job runs during the night when I don't use KooZic. There is no concurrent update issue then. But ok, good to know, I fixed it at https://github.com/DocMarty84/oomusic/commit/a89384ccbbb5a84aac2a3d36f7cab517948a03d0

By the way, I fined tuned a couple of things in https://github.com/DocMarty84/oomusic/commit/0ca6316325b2e09c072b27480aa6606a34c74b63 to make the cache mechanism much less aggressive.

saw a few things, just after deleting a playlist music was playing, suddenly playing stops during playback and starts at beginning again.

Did you delete the playlist you were playing? Did the traceback you wrote arose at this very moment?

music playing, suddenly stops and next song in playlist is started

That one is really interesting for me. Are you using Firefox? If so, which version? What was the length of the song? Did it happen after like ~5-6 minutes of playing into the song? I'm asking that since I am facing a similar issue since Firefox 57. For "long" songs (above 5-6 minutes), it regularly happens that the song stops playing and the next song in playlist is started. I never checked the log at that moment, I always assumed that Firefox was doing some crap, but I was maybe misleading... But still, I didn't face the issue with Chrome.

aroundmyroom commented 6 years ago

Chrome I use length of song was around let me see 4 of 6 mins, stopped around 3 minutes. But I mostly use only mix/remix/12-inch/megamix music +10 minutes etc..

But last week I was playing a mix of around 60 minutes over VPN looked and listened just fine

Will keep track of it. if there is a way to use git pull let me know

after 4.29 the song of 16.00 was cut off and next song was started

image

logfile did not show directly an issue, that came a little later when skipping through the same song again

image

aroundmyroom commented 6 years ago

yep same song now after 4.17

Apr 22 19:38:38 koozic odoo-bin[1660]: 2018-04-22 17:38:38,223 5498 INFO koozic werkzeug: 10.1.1.101 - - [22/Apr/2018 17:38:38] "POST /web/dataset/call_kw/oomusic.playlist.line/search_read HTTP/1.1" 200 - Apr 22 19:38:57 koozic odoo-bin[1660]: 2018-04-22 17:38:57,451 5543 ERROR koozic odoo.service.server: Worker (5543) Exception occured, exiting... Apr 22 19:38:57 koozic odoo-bin[1660]: Traceback (most recent call last): Apr 22 19:38:57 koozic odoo-bin[1660]: File "/opt/koozic/odoo/service/server.py", line 734, in run Apr 22 19:38:57 koozic odoo-bin[1660]: self.process_work() Apr 22 19:38:57 koozic odoo-bin[1660]: File "/opt/koozic/odoo/service/server.py", line 767, in process_work Apr 22 19:38:57 koozic odoo-bin[1660]: self.process_request(client, addr) Apr 22 19:38:57 koozic odoo-bin[1660]: File "/opt/koozic/odoo/service/server.py", line 758, in process_request Apr 22 19:38:57 koozic odoo-bin[1660]: self.server.process_request(client, addr) Apr 22 19:38:57 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 318, in process_request Apr 22 19:38:57 koozic odoo-bin[1660]: self.finish_request(request, client_address) Apr 22 19:38:57 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 331, in finish_request Apr 22 19:38:57 koozic odoo-bin[1660]: self.RequestHandlerClass(request, client_address, self) Apr 22 19:38:57 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 654, in init Apr 22 19:38:57 koozic odoo-bin[1660]: self.finish() Apr 22 19:38:57 koozic odoo-bin[1660]: File "/usr/lib/python2.7/SocketServer.py", line 713, in finish Apr 22 19:38:57 koozic odoo-bin[1660]: self.wfile.close() Apr 22 19:38:57 koozic odoo-bin[1660]: File "/usr/lib/python2.7/socket.py", line 283, in close Apr 22 19:38:57 koozic odoo-bin[1660]: self.flush() Apr 22 19:38:57 koozic odoo-bin[1660]: File "/usr/lib/python2.7/socket.py", line 307, in flush Apr 22 19:38:57 koozic odoo-bin[1660]: self._sock.sendall(view[write_offset:write_offset+buffer_size]) Apr 22 19:38:57 koozic odoo-bin[1660]: timeout: timed out Apr 22 19:38:57 koozic odoo-bin[1660]: 2018-04-22 17:38:57,623 5555 INFO koozic odoo.service.server: Worker WorkerHTTP (5555) alive Apr 22 19:42:55 koozic odoo-bin[1660]: 2018-04-22 17:42:55,962 5555 INFO ? odoo.http: HTTP Configuring static files Apr 22 19:42:55 koozic odoo-bin[1660]: 2018-04-22 17:42:55,995 5555 INFO koozic odoo.addons.base.ir.ir_http: Generating routing map

DocMarty84 commented 6 years ago

Well, I can't reproduce the traceback (maybe because I'm using a development version), but I can definitively reproduce skipping to the next song. Although in my case, it always happen after ~15 minutes, and this might be due to Nginx (which I am using on top of KooZic) which closes the idle connection.

fail at 15minutes

Every time you see seek=... in the URL, it means that the track restarted from this point. It's always around 900 seconds (15 minutes).

I'm at home with a good connection, but when at work (with a pretty bad network quality), it can fail after 5-6 minutes. KooZic should be able to handle this: you're not always in an environment with a perfect connection, so the transfer of the song might face an error at some point. In that case, it should be able to get back on its feet and restart from where it failed.

I have a workaround for the issue at https://github.com/DocMarty84/oomusic/commit/be62e7cedce36414e74c32bf63ccac84b10074f3 I'll be testing it at work tomorrow and update you if it seems reliable enough. The idea is that if the song stops before the end, we restart from the last "correct" status we got from it. Since we get a status every 3 seconds, you might hear the song stop and a slight replay of what you just heard (max 3 seconds). If it happens once in a while only on long songs, it should be acceptable.

DocMarty84 commented 6 years ago

Sorry for the late reply.

First, I was able to find a workaround for the playback errors (https://github.com/DocMarty84/oomusic/commit/9721a3cdf239820a2397255aee6b0355c9161ba2). Moreover, I've been testing several combinations with the following options:

Option 1: transcoding activated, HTML5 Audio

This is the default configuration, and probably the best compromise. With transcoding activated, the browser always receives the same format (OPUS), and the HTML5 Audio allows starting playing the track even if it is not fully downloaded. Switching from one track to another is fast. The only drawback is that HTML5 seems slightly less reliable than web audio. On long tracks, some playback errors might occur.

Option 2: transcoding deactivated, HTML5 Audio

Lightest configuration for the server. No need of FFmpeg since transcoding is deactivated, and HTML5 allows starting the playback quickly. In case of playack error, resuming is fast. The drawback is that web browsers might not handle all audio formats correctly. From my experience, Firefox is great with OGG, Flac or MP3. Chrome, on the other hand, is great with OGG, but doesn't support all features for FLAC and MP3. For example, pausing doesn't work.

Option 3: transcoding activated, Web Audio API

This is the safest configuration, but potentially the slowest. With transcoding activated, the web browser always receives the same format. Moreover, Web Audio downloads the whole track before playback, making it very reliable. I didn't face any playback error, even on very long tracks. The major drawback is the loading time between tracks: since you need to transcode and receive the whole track, the performances of your server are critical for smooth transitions.

Option 4: transcoding deactivated, Web Audio API

This configuration is perfect if your server has a fast connection, since you need to download the whole file before playing. It mixes the pros and cons of options 2 and 3, so it depends how you are affected.

Conclusion

There is no "perfect" option, so I leave the configuration open. It will be up to the user to test the one that suits best the set-up. For me, the best is option 4. I am using Firefox, so no problem playing all audio files. My server connection is very good, but the connection at work is quite unreliable, leading to a lot a playback errors when I use HTML5 Audio (which doesn't happen at home, where my connection is way more stable).

DocMarty84 commented 6 years ago

If you want to try all these options, the v1.0.0-beta.3 is out!

To install, quite easy:

curl https://raw.githubusercontent.com/DocMarty84/koozic/11.0/extra/installer/koozic_install.py > k_install.py
chmod +x k_install.py
sudo ./k_install.py install

Although, I would suggest to install on another VM, to avoid conflicts. Debian 9 is still supported.

aroundmyroom commented 6 years ago

I am this week in Denmark will test next weekend

Verstuurd vanaf mijn iPhone

Op 1 mei 2018 om 22:08 heeft DocMarty84 notifications@github.com het volgende geschreven:

If you want to try all these options, the v1.0.0-beta.3 is out!

To install, quite easy:

curl https://raw.githubusercontent.com/DocMarty84/koozic/11.0/extra/installer/koozic_install.py > k_install.py chmod +x k_install.py sudo ./k_install.py install Although, I would suggest to install on another VM, to avoid conflicts. Debian 9 is still supported.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

DocMarty84 commented 6 years ago

Thanks, do not hesitate to report any issue.

aroundmyroom commented 6 years ago

I do not see any progress with my very large database. the whole UI is slow (I did setup a clean Debian 9 with new installation) ..

What prevents me to use this is

Thanks

DocMarty84 commented 6 years ago

That's very weird... Slow UI, I could still understand, but the drop-out is surprising since this should be prevented now. Can you confirm that you tried: curl https://raw.githubusercontent.com/DocMarty84/koozic/11.0/extra/installer/koozic_install.py > k_install.py

(11.0 in the URL is important)

To be sure of the version you have, you should now see statistics for a folder you added (number of tracks, albums, artists...). Go to Configuration > Folders, and on the folder you added you should see the stats. Is this correct?

aroundmyroom commented 6 years ago

image

Yep I have that Just played 'Giorgio Moroder's - Battlestar Galactica (16min) after 4.26 a little drop-out. and at 8.52 as well ..

So for me its unfortunately not an improvement to go away from Subsonic ..

DocMarty84 commented 6 years ago

OK, I didn't get what you meant by drop-out at first, I though you still had the song skipping to the next one.

4.26 a little drop-out. and at 8.52 as well ..

That's... surprising. A drop-out after exactly 4:26, twice in a row.

Anyway, the drop-out is the only workaround I came up with when using HTML5 Audio. At maximum, it can be a drop-out of 3 seconds. It could be possible to reduce it, but not possible to get rid of it at the moment.

However, with Web Audio, there should not be any drop-out. I've been testing it extensively for the past 2 weeks, and I didn't face any drop-out. See my comment at https://github.com/DocMarty84/koozic/issues/7#issuecomment-385178456 , you could test options 3 or 4. The only annoying part is the waiting time between tracks since you need to download the whole file before playing. If you let the player run, that's mitigated by the fact that the next track is pre-fetched 30 seconds before the end of the current track. If you go though a playlist manually (by clicking 'Next'), then it's obviously not the case.

I'm still surprised that you mention a slow UI. Is it still as slow as before (with high CPU/Memory usage)? Because there were several changes:

Basically, it's one batch of 500 folders/artists every night. In my case, running a batch of 500 folders takes 3 minutes with a limited memory footprint. So even if it would be run during the day, it should not be a major issue.

Still, there is something I can fine-tune to avoid filling-up more memory than available. When installing KooZic, I always assume that memory is not a limitation. I should probably be more careful about this.

Thanks for the report anyway, that's very helpful.

aroundmyroom commented 6 years ago

no drop-out of the same song after around 4.18 ~ 4.26 every song > 5 min or so. drop out was around <1 to 1,5 seconds. I played it within Google Chrome (web audio or html5 ?)

yes the browsing is slow. it can take up to 20 to 30 seconds switching from 'albums' to artists and sometimes it seems to stall the screen 'loading, still loading, please be patient' and that part is soooo frustrating ;) As I think its all database driven it should be 'instantly' be able to be displayed. Yes I have +28000 artists in the db but if you load 80 in the first page it should be much more quicker

DocMarty84 commented 6 years ago

As I think its all database driven it should be 'instantly' be able to be displayed. Yes I have +28000 artists in the db but if you load 80 in the first page it should be much more quicker

The very first time you access the albums or artists page, it can take some time. The albums page will compute all the thumbnails, but they are computed once and for all. The next accesses are then much quicker. Regarding the artists page, this can be slower because images are retrieved from LastFM. However, once again, it is only the very first time you reach the page, afterwards the thumbnail is saved.

I create a Debian 9 VM with 1 CPU and 1GB of RAM. I simulated a large music collection thanks to several SSHFS mounting points linking to my music collection:

screenshot-2018-5-12 false - odoo

The only thing I can't reproduce is the number of artists.

I noticed that during the scan of the music library, KooZic uses quite a lot of memory. When the system starts, the total memory consumption is ~150 Mo. When the collection is scanned, it grows up to 650-700 Mo. There are several reasons to this:

  1. KooZic caches the data in memory, for faster comparison (access to a DB is a bottleneck)
  2. PostgreSQL itself caches data as well
  3. SSHFS uses some extra memory

If you add some extra services, it would not be surprising to hit the 1GB memory limit, start swapping and... be super slow. That's why I'm pretty sure it comes from some memory overload.

Regarding point 1, I made a first modification to compute in a smarter way the memory limits allowed: https://github.com/DocMarty84/koozic/commit/e3e79cc72da028c8de6d349369991ed1d6ce46ba Indeed, I noticed that the default value was 2 Go / process before it was cleared (here). Obviously in your case that's way too high. From now on, the limit will take into account the physical memory available. For your config, I set up the limit to be 300 Mo / process with 3 processes maximum running. So the KooZic processes should never use more than the physical limit.

I still need to investigate point 2. Fine tuning a DB is not a piece of cake.

However, that seems promising. With such a setup, I never go over 700 Mo used for the whole system, playing around with KooZic, browsing albums, artists, genres, tracks, scanning, etc.

For the extreme cases, I will also make a magic 'reduce CPU' button. That will:

The latter is important for smooth transitions. You mentioned that switching from albums to artists was slow, but you didn't mention that the tracks view was slow, although you have ~144k tracks and 'only' 19k artists.

Hopefully you'll be able to have a smooth experience with KooZic.

DocMarty84 commented 6 years ago

Finally, I went with 3 settings:

  1. (De)activate the background jobs, including the automated scanning
  2. Set the default view: Kanban (thumbnails) or list (you can switch from one to the other at the top right, next to the pager)
  3. LastFM infos fetched automatically or manually (on an artist: Action > Update LastFM Infos)

I think options 2 and 3 will be have the most noticeable performance impact when using albums or artists views.

I will probably publish a new beta tomorrow with all these features included.

Related commit: https://github.com/DocMarty84/oomusic/commit/27458d57c88c0ca8f5392177d5b06930f3c286fd

DocMarty84 commented 6 years ago

I've uploaded a new beta release. The new options:

screenshot-2018-5-13 settings - odoo

To upgrade:

curl https://raw.githubusercontent.com/DocMarty84/koozic/11.0/extra/installer/koozic_install.py > k_install.py
chmod +x k_install.py
./k_install.py upgrade

This will allow upgrading without having to to scan you whole collection. However, please note that the systemd file won't be updated with the memory fine tuning. In your case, you can modify /etc/systemd/system/koozic@.service: ExecStart=/opt/koozic/odoo-bin -d koozic --limit-time-real=3600 --limit-time-cpu=1800 --workers=2 --limit-memory-hard=1073741824 --limit-memory-soft=348285610 --max-cron-threads=1 Then:

One last remark: if your VM has only 1 CPU, ffmpeg might also be the culprit of a slow interface. Although it might seem obvious, but when transcoding a track, ffmpeg uses 100% of the CPU. In case of a single CPU, that won't let much resources to other processes... Note that you can deactivate transcoding on a playlist.

aroundmyroom commented 6 years ago

I will try it, compared to your installation my koozic is maybe 10 or 20 times slower than your demo

but when I use https://github.com/IrosTheBeggar/mStream This one is very very quick !

VM has 1 CPU, 1024MB, more is not acceptable for what I want.

DocMarty84 commented 6 years ago

Hopefully, the new options will make it faster. It's basically removing a bunch of fancy stuff that are not mandatory and may slow down the interface. Please note that since KooZic is compatible with the Subsonic API, it is also compatible with Chrome extensions (e.g. SubFire). However, I don't use any of them, so I don't know how reliable they are.

but when I use https://github.com/IrosTheBeggar/mStream This one is very very quick !

Yep I can imagine, that's a different approach. That being said, that would be nice to know if it also faces drop-out. Indeed, it uses the same library than KooZic (howler.js, same version 2.0.9), and uses the HTML5 option which was causing a drop-out. The only difference I see is that it converts to MP3 by default with FFMpeg, while I convert to Opus by default. I will check if I face drop-outs with MP3 conversion by default as well.

DocMarty84 commented 6 years ago

By the way, from a quick test, I can see that mStream doesn't convert my FLAC files when they are played in the browser. I haven't really dug in the options to see if this can be changed, though. Anyway, they use the same configuration than Option 2 described here.

aroundmyroom commented 6 years ago

no issue with drop-out with mStream (no subsonic api yet) ;) but it does right now what I am expecting. Be quick and give me the songs I need in a flash.

aroundmyroom commented 6 years ago

btw it took some waiting time, but the initial browse on the first page is now quick, I disabled all processes like in your example but than when I press an album it takes still 30 to 40 seconds to show the next page.

aroundmyroom commented 6 years ago

I will abandon koozic but will keep eye to it. The tool is not 100% fit for my purpose. it is too slow for what I want with a 4TB large database. function wise it has all I would need or like, but the chosen framework is not what I really like.

As I wrote earlier I move towards mStream. Although feature wise it has less function, but in a system where I do not care about tags or albums and where my music catalog is devided in subfolders it matches my need(s). if subsonic can be supported it would be the 100% match ;)

DocMarty84 commented 6 years ago

but than when I press an album it takes still 30 to 40 seconds to show the next page.

That's definitively weird... Honestly, there is not much I see that could impact the performances so much. Only thing would be FFMpeg running in the background and using all the CPU, but you can deactivate transcoding. So without being able to reproduce the behavior, that's difficult to find the root cause.

in a system where I do not care about tags or albums and where my music catalog is devided in subfolders

The browsing mode is very fast in koozic, but very limited as well. Search isn't even available, and that mode was introduced for convenience, not intended to be used only as itself. Koozic was designed to make extensive use of tags, bringing a lot of searching capabilities and sorting. If the purpose is to browse your directories, mStream or CherryMusic might be better alternatives.