The recorder configuration parameter rollover_idle_secs, as described here does not seem to be working properly, with any value.
Steps to reproduce the bug
Configure a recorder with a configuration such as:
recorder:
source_coll: patching_source
rollover_size: 100000000
rollover_idle_secs: 600
filename_template: patching-{timestamp}-{hostname}-{random}.warc.gz
source_filter: live
After performing several recordings, several warc files will be present on the filesystem with a size below rollover_size. After waiting for more than 600 seconds and performing another recording, the same warc files are still open and being written to.
Expected behavior
After waiting for more than rollover_idle_secs before performing a recording, a new set of warc files should be opened for writing.
Describe the bug
The recorder configuration parameter rollover_idle_secs, as described here does not seem to be working properly, with any value.
Steps to reproduce the bug
Configure a recorder with a configuration such as: recorder: source_coll: patching_source rollover_size: 100000000 rollover_idle_secs: 600 filename_template: patching-{timestamp}-{hostname}-{random}.warc.gz source_filter: live
After performing several recordings, several warc files will be present on the filesystem with a size below rollover_size. After waiting for more than 600 seconds and performing another recording, the same warc files are still open and being written to.
Expected behavior
After waiting for more than rollover_idle_secs before performing a recording, a new set of warc files should be opened for writing.
Screenshots
Environment
centOS 7 pywb version 2.6.6
Additional context