search

QuiltServerTools / Ledger

A server-side logging mod for fabric 1.17+
https://www.quiltservertools.net/Ledger/latest/
GNU Lesser General Public License v3.0
190 stars 52 forks source link

Faster Database queue draining #164

Open SulpherStaer opened 2 years ago

SulpherStaer commented 2 years ago

Is your feature request related to a problem?

Database queue is still draining. If you exit now actions WILL be lost

At the moment, the database queue does not drain fast enough on shutdown and the server gets cut off by the watchdog saying it took to long to shut down (presumed crash) and kills it after 60s. Ledger is not fast enough to handle this because of reasons?

Describe the solution you'd like.

I'd like Ledger to more quickly drain the queue so I do not lose any actions during restarts.

Describe alternatives you've considered.

I think more connections to the mysql database should be made to clear the queue more quickly. Another plugin I use (Plan, Player Analytics) uses a configurable 'max connections' that it will use to save its data.

I have the feeling Ledger just uses ONE connection to the mysql, because it is Sooooo slow to do any inspection or saving.

Agreements

Other

No response

object-Object commented 1 year ago

Is there any update/progress on this? For context, I'm working on setting up a custom pack/server (GitHub). It's a quite technical community, so I'm expecting a fair amount of large-scale projects such as world eaters/terraforming. We ran a (somewhat) small test and it hadn't finished processing half an hour later when we restarted the server. It doesn't help that /ledger status gives no information about the size of the queue. I'd really like to be able to use Ledger, but taking hours to process the queue isn't very tenable.

MeesJ commented 1 year ago

Ledger's MySQL connection seems to be as inefficient as it can get. Not only the database queue draining is slow, but the inefficient database connection is also causing gaps in the logs the mod is supposed to collect.

Ledger should use a pool of connections instead of just one single, inefficient connection. That's what CoreProtect does as well.

JosTheDude commented 12 months ago

Any update on this? Doesn't look like there's been of a migration over to a connection pool but it'd significantly lessen the performance hit of logging overall.

Database drain is a significant part of our server's shutdown time, so.