Closed irregulator closed 5 months ago
@austb Hi! Maybe you have insight regarding this issue? Or perhaps you can ping someone who has. Thanks in advance.
@irregulator Running multiple PuppetDB servers connecting to a single Postgres is a supported use case, and it is done in many Puppet Enterprise installations.
In current versions of PuppetDB, errors like that are expected, and will even be generated when you have a single PuppetDB because it can submit multiple commands at the same time. The reason you see those errors in Postgres and do not see any errors in PuppetDB is because we catch that database error and automatically retry. You will only see those errors in PuppetDB if the retry attempts fail as well.
The specific SQL causing the error you are seeing was also causing a performance impact when we did internal scale testing simulation 100,000 agents, so a recent commit should reduce the prevalence of that error in the logs. But, it would still be considered normal to see ERROR: could not serialize access due to concurrent update
in the Postgres logs due to the concurrent submission of data. If you do not see the error in the PuppetDB logs, it means the retry was successful and your data was correctly stored in the database.
@austb thanks for the elaborate answer.
Just to be clear, we are running the non-enterprise version of PuppetDB, provided by https://apt.puppet.com/ . Can we safely assume that the "multiple PuppetDB against a single Postgres" holds true in that case as well?
Thanks and let's close the issue afterwards.
Yes, non-enterprise PuppetDB has the same multi-PuppetDB -> single Postgres capabilities as the enterprise version.
Describe the Bug
Hello and thanks for developing PuppetDB!
Based on this sentence:
we are testing a setup with two (or more) PuppetDB instances using the same PostgreSQL database.
So let's assume the following:
pgsql001
hosts the PostgreSQL database namedpuppetdb
puppet001
runs bothpuppetserver
andpuppetdb
and points topgsql001
databasepuppet002
runs bothpuppetserver
andpuppetdb
and points topgsql001
databaseBoth PuppetDB instances are employing the very same credentials against a single database.
puppet001
andpuppet002
are used by a different set ofpuppet-agents
, for example hosts in two different datacenters.puppetdb
logs feature no error butpostgresql
logs do:This seems like the two PuppetDB instances are competing to update the same row.
So our questions are:
Expected Behavior
We would expect to be run two different PuppetDB instances without database errors.
Steps to Reproduce
Steps to reproduce the behavior:
Environment
8.3.0-1jammy
8.4.0-1jammy
14.10-1.pgdg22.04+1
Additional Context
I will gladly provide more information or context on this topic if requested.