rembo10 / headphones

Automatic music downloader for SABnzbd
GNU General Public License v3.0
3.36k stars 602 forks source link

Draft PostgreSQL support #3232

Open kainz opened 4 years ago

kainz commented 4 years ago

So I've had a branch with PostgreSQL support for headphones sitting around for a couple years now, thought I'd update it.

Why did I do such a thing?

I would not call this a clean implementation, it has a few glaring issues:

I'd like to take some of the bits I've identified here, and maybe combine them to add SQLAlchemy support and get something close to real cross-db compatibility. Is there any interest upstream in such a feature?

At least in my local use case, this work has given me a definitely better experience running headphones, as I have no (sqlite) locking delays between the various threads headphones runs. Some queries/searches can also be faster, but I haven't dived really deep into this for awhile.

This branch can also run successfully on pypy, but I have not benchmarked it in awhile to see how different the speed would be.

Currently, I've been testing this branch on pypy 2.7 (pypy7.0.0), and PostgreSQL 9.5. I'm of a mind to upgrade to 9.6+ though, as 9.6 adds alter table add column if not exists syntax.

Thoughts?

monik3r commented 4 years ago

Hi,

I've been looking into this branch for testing, I have a pretty large headphones instance and it tends to grind a bit. I've attempted to use this dockerfile: https://gist.github.com/monik3r/a7f68676582c5e4837702ddfff9a79c0 , but the server won't not reset the connection on each attempt.

Do you have any details on how to set this up? Should I be setting up a postgres db with some form of creds?

Thanks, it would be really neat to try out your code (and benchmark it)!

Edit: checked the head of the logs and found an error that was being returned from the app, I'll keep looking into it, here's the log: https://gist.github.com/monik3r/0ec924638bfe8b4311c7ccaf379f0a1c

Edit 2: Managed to get everything working, had to hardcode in some server values for now to get the webserver started, as well as change localhost to 0.0.0.0 - thee were likely caused by me during deployment. I'll let it run for a few hours and see how it performs compared to the current maser headphones