search

darold / pgbadger

A fast PostgreSQL Log Analyzer
http://pgbadger.darold.net/
PostgreSQL License
3.55k stars 352 forks source link

Enhancement : Add possibility to rebuild only some reports #689

Closed thomasLeclaire closed 2 years ago

thomasLeclaire commented 3 years ago

Hello, a nice feature could be to allow to rebuild some reports related only to some part of binaries. I keep 3 months of binaries, but rebuild all on this data is too long and use too much memory.

on the same time add some error management for the corrupted files and just inform and continue the process could prevent to have to restart after manual cleaning.

regards

darold commented 3 years ago

You cab use --month-report=YYYY-MM to only rebuild reports from a given month, it should be enough no?

Commit 21db60b should handle your suggestion about skipping failure on binary files.

thomasLeclaire commented 3 years ago

thx for the skip failure!

about --month-report, in fact in my case, process one month of data is too much, i already have difficulties to process only one week. (8cpu, 64G ram, but around 250G logs each day..). so be able to load only a specific part of the binaries files could make sense.

darold commented 3 years ago

Commit b20c6c7 adds new option --day-report "YYYY-MM-DD" that can be used to rebuild an HTML report over the specified day. Like option --month-report but only for a day. I hope it will be helpful in your case, let me know.

thomasLeclaire commented 2 years ago

still some strange stuffs like it load really more than the mandatory data, but sounds like a more global issue not related to this feature. I will open new ticket if I am able to find the rootcause. Thx anyway for the nice enhancement!