Persistent cache not working?

[gforcada]

Versions

python==3.8 RelStorage==3.5.0 psycopg2-binary==2.9.8 PostgreSQL==15.4 Plone==5.2.12

RelStorage configuration

The RelStorage configuration on Plone looks like this on zope.conf:

<relstorage>
    keep-history false
    blob-dir var/blobstorage
    blob-cache-size 2gb
    cache-local-mb 200
    cache-local-dir var/relstorage-cache
    <postgresql>
        dsn dbname='datafs' user='relstorage' host='aa.bb.cc.dd' password='xxx'
    </postgresql>
</relstorage>

From what I read on the documentation, I expected that inside the cache-local-dir an sqlite database would be created and cache data being persisted there across restarts.

Turns out that the file gets created but it's rather empty:

-rw-r--r-- 1 s-derfreitag service  16K Nov 10 14:58 relstorage-cache2-datafs.sqlite3

And on the Plone instance logging there is no mention of having a problem, there are only INFO messages regarding RelStorage:

2023-11-10 07:21:31,868 INFO    [relstorage.storage.tpc:131][waitress-1] Using store connection <StoreConnection at 0x7faab8196760 active=False description={'backend_pid': 102945} conn=<connection object at 0x7faaa50f1900; dsn: 'user=relstorage password=xxx dbname=datafs host=aa.bb.cc.dd application_name='RS: Store'', closed: 0> cur=<cursor object at 0x7faaa4b26e50; closed: 0>>

Side question: That within a day, and only having restarted the server one, maximum twice, I got +50 messages like the above is not a problem either, right? 🤔

Any ideas, what's wrong with my setup to have an empty sqlite file?

[jamadden]

Any ideas, what's wrong with my setup to have an empty sqlite file?

The database isn't being closed before the process exits. RelStorage writes to its cache when it is closed.

[gforcada]

Thanks for the prompt answer! 🙇🏾

When a Plone client is restarted, it should close the connection to the database, and that's when RelStorage should be writing its cache, right?

Then somehow Plone is not closing the connection properly? 🤔

[mamico]

@gforcada some time ago I made this fix in Zope for this purpose https://github.com/zopefoundation/Zope/pull/740 I suggest you check if when your instance stops, this code is executed.

[gforcada]

Turns out that locally it worked with Plone:

./bin/instance fg

And when stopping it with Ctrl+C it was writing the file.

But the log files on production did not show those messages.

We are using supervisord to manage the process, we instruct supervisord to start Plone with ./bin/instance console and then when it restarts it sends the configured stopsignal, which by default is SIGTERM.

Changing that to SIGINT seems to fix it 💫

[gforcada]

@mamico nice 😄 I just found another way to solve it a few minutes ago ⏰

We are still using Plone 5.2.x so that Zope version is not in our version pins unfortunately. Hopefully we can move soon to Plone 6 🎉