photoview / photoview

Photo gallery for self-hosted personal servers
https://photoview.github.io/
GNU Affero General Public License v3.0
5.33k stars 398 forks source link

Needs many DB connections when photos are from different months #688

Open m3nu opened 2 years ago

m3nu commented 2 years ago

Describe the bug I was getting errors about too many connections for both MySQL and Postgres. For MySQL my server limit was set to 45 connections, which is less than the 24 specified in the source code here: https://github.com/photoview/photoview/blob/f1ce85dcb591016973b2c156de4e7328015e822d/api/database/mysql.go#L43

The only DB option that worked without increasing the limit a lot was SQLite.

My test import had photos from many different months. Is it possible that Photoview opens one connection per month, without respecting the limite.

To Reproduce

Expected behavior There should be a connection limit that's respected. Else users need to increase the limit, the more pictures they add.

Your environment:

monotok commented 2 years ago

I just thought i'd give photoview a try again and I came across this exact issue. I noticed this after I refreshed the timeline page during an import of photos. Photoview was busy scanning a collection of around 30K. I have it connected to a remote (another VM) postgres DB which is also the DB for other applications. After the scan was complete it went away on its own.

afrojewelz commented 2 years ago

even pgpool cannot handle those many connection ,i wonder if this ORM uses any recycleable /multiplex pool instance?

my pgsql-ha just went nuts when scan all my photos failed to connect to host=10.43.50.2 user=postgres database=photoview: server error (FATAL: Sorry, too many clients already (SQLSTATE 53300))