search

zodb / relstorage

A backend for ZODB that stores pickles in a relational database.
Other
54 stars 46 forks source link

Improve logging for zodbconvert and with psycopg2 #426

Closed jamadden closed 3 years ago

jamadden commented 3 years ago

zodbconvert logs more useful data at DEBUG level and drops some things to TRACE. A new CLI argument can enable TRACE. Regular logging includes more data too.

Avoid logging the warning about not being in a transaction during normal TPC when using psycopg2. Deal gracefully with psycopg2 Connection.notices of None.

Example zodbconvert logging at DEBUG:

INFO   Copied transaction 0x03cba93d6dab2211 2018-11-16 09:01:25.703552 <   2 records,   0 blobs,   5.02 KB> |     6211/6932754,  0.09%, 182.90 KB/s 103.497 TX/s, 10.72 MB | (259766/6932754,  3.75%, 922.52 KB/s 76.206 TX/s, 3070.90 MB 56.8 minutes)
DEBUG  Copied transaction 0x03cba94182012622 2018-11-16 09:05:30.469802 <   1 records,   0 blobs,   1.17 KB> in 0.0163s
DEBUG  Copied transaction 0x03cba94e3873c1dd 2018-11-16 09:18:13.230979 <   1 records,   0 blobs,   2.13 KB> in 0.0078s
DEBUG  Copied transaction 0x03cba95f60f3ccee 2018-11-16 09:35:22.723206 <   1 records,   0 blobs,   1.00 KB> in 0.0082s
DEBUG  Copied transaction 0x03cba96e5c3f85dd 2018-11-16 09:50:21.620657 <   1 records,   0 blobs, 419 bytes> in 0.0099s
INFO   Copied transaction 0x03cba9769f0ce022 2018-11-16 09:58:37.277413 <   2 records,   0 blobs,   2.92 KB> |      5955/6932754,  0.09%, 190.70 KB/s 99.248 TX/s, 11.17 MB | (265721/6932754,  3.83%, 909.86 KB/s 76.605 TX/s, 3082.08 MB 57.8 minutes)