pulsejet / memories

Fast, modern and advanced photo management suite. Runs as a Nextcloud app.
https://memories.gallery
GNU Affero General Public License v3.0
3.23k stars 87 forks source link

Rebuilding places database breaks at version 7.0.2 #1080

Closed echtfrank closed 8 months ago

echtfrank commented 8 months ago

Describe the bug After migrating from version 7.0.1 to version 7.0.2, rebuilding the places database repeatedly breaks at polygon 1250232_0.

To Reproduce Login to CLI and run command:

ncc memories:places-setup ("ncc" is equivalant for "occ" on Nextcloud implementation on DietPi)

Screenshots Last lines of logging:

Inserted 112000 / 635189 places (17.6%), Last: Sarasota County Inserted 112500 / 635189 places (17.7%), Last: 沿溪镇 Inserted 113000 / 635189 places (17.8%), Last: Климовский район Inserted 113500 / 635189 places (17.9%), Last: Nanzdietschweiler ERROR: Failed to insert polygon 1250232_0 (An exception occurred while executing a query: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away In ExceptionConverter.php line 104: An exception occurred while executing a query: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away In Exception.php line 28: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away In Statement.php line 130: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away memories:places-setup [-f|--force] [-r|--recalculate] [--transaction-size TRANSACTION-SIZE]

Platform:

Additional context Yesterday, after upgrading from Memories 7.0.0 to 7.0.1 I performed the exact same step, and this went well.

ExceptionFailed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [2002] No such file or directoryException thrown: Doctrine\DBAL\Exception

echtfrank commented 8 months ago

I did an additional re-indexing of all files using the command occ memories:index --retry. After this I rebuild the places database again.

It is now running a lot further:

Inserted 513500 / 635189 places (80.8%), Last: Lucaston ERROR: Failed to insert polygon 10627494_0 (An exception occurred while executing a query: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away

But unfortunately, it does not finish properly and exits again. It seems like the database is not available? Anyway, I think it is strange that yesterday I had no problems, and now I have... given the fact that version 7.0.2 so quickly follows the 7.0.1-release.

pulsejet commented 8 months ago

https://memories.gallery/troubleshooting/#general-error-2006-mysql-server-has-gone-away

You don't actually need to re run this for the patch versions. Just running it once is good enough

echtfrank commented 8 months ago

Thanks, for pointing me to the trouble shooting guide. I am going to investigate and try further.

Cyberes commented 8 months ago

My install warned me that Reverse geocoding has not been configured (-1) so I ran occ memories:places-setup --transaction-size=5.

I got the 2006 MySQL server has gone away error and adjusted my mariadb config settings to increase the timeout:

max_allowed_packet = 256M
net_read_timeout = 60
net_write_timeout = 120
wait_timeout = 28800
innodb_log_file_size = 256M

innodb_buffer_pool_size = 5G

This did not fix the issue and I got the error again. I watched the process and the MySQL server isn't overloaded.

Update: max_connections was set too high. Lowering it fixed the issue.

pulsejet commented 8 months ago

I'm assuming this has been fixed. Please reopen otherwise.