WAL file keeps growing until server is restarted

[jgiannuzzi]

We are experiencing checkpoint starvation, as described in https://www.sqlite.org/wal.html#avoiding_excessively_large_wal_files. In combination with #445, this means that a FastTrackML server running on Kubernetes with the SQLite backend will have an ever-growing WAL file and will quickly run into disk space issues!

[suprjinx]

perhaps starting the server goroutine with a timeout context, which makes it periodically exit -- then do the truncate and restart (in a loop)?

[jgiannuzzi]

I got a POC running that simply tunes the connection pools so that we don't keep idle connections constantly open and this works fine — however I want to make sure I understand the potential performance implications of that

[dsuhinin]

perhaps starting the server goroutine with a timeout context, which makes it periodically exit -- then do the truncate and restart (in a loop)?

ohh, it sounds like a NodeJS application :)

[suprjinx]

@jgiannuzzi should we PR your branch?