Abnormal memory usage in Postgres

sfa-siard / SiardGui

SIARD Suite - SiardGui

Other

17 stars 3 forks source link

Abnormal memory usage in Postgres #60

Open audunlund opened 1 year ago

audunlund commented 1 year ago

Hello, Downloading a Postgres DB with mainly pdfs stored in data type bytea needs abnormal amount of memory. To download a 3.5 GB SIARD File, SIARD Suite has to be started with Xmb11g and uploading the same SIARD File is even worse. Then I have to start the SIARD Suite with max memory of 13GB. Thanks in advance. Best regards Audun

mburri commented 1 year ago

Hi @audunlund thank you for the issue. Did this behave differently with earlier 2.1.x versions of the SIARD Gui?

audunlund commented 1 year ago

Hello mburri, I don’t know, but support for Postgres is pretty new and Postgres is not (yet) the most widespread database due to archiving so I doubt that this constellation (Loads of pdf in bitea datatype) has been tried out before. An agency is about to deliver an extract from a database like that. This DB contains protected data so I can’t deliver that one to you, but I have reproduced the behavior myself and I would be happy to provide you with this SIARD file or even better: Inform you how to reproduce this behavior yourself. Best regards Audun

mburri commented 1 year ago

We just confirmed that this issue was already present in SiardGui 2.1.x

And at least for MS Sql Server archiving a large database >4GB with the original memory settings works as expected.

The problem is specific to PostgresQL and affects all versions of the Siard Suite