search

m-lab / etl

M-Lab ingestion pipeline
Apache License 2.0
22 stars 7 forks source link

One file failing to process #190

Open gfr10598 opened 7 years ago

gfr10598 commented 7 years ago

"gs://archive-mlab-oti/ndt/2017/06/28/20170628T000000Z-mlab1-ord03-ndt-0077.tgz"

gfr10598 commented 7 years ago

It contains a single uncompressed s2c test file that is 2.1 GB, and two small associated files. It appears that this single large test is perhaps causing a crash?

gfr10598 commented 7 years ago

Yes - looking at all logs- there is an OOM for the process that receives the request.