Open JustAnotherArchivist opened 1 year ago
That is indeed what causes these crashes. One job in particular produced lines of up to 1.7 MiB. The buffer is only 1 MiB. The fix here is probably to drop lines that exceed some limit. Whether that should be 1 MiB or larger, I'm not sure, but really that size ought to be sufficient.
This crash happened twice today:
Sounds like overlong lines from the firehose cause this crash, potentially a job with very long URLs.