search

jwbensley / dfz_name_and_shame

DFZ Name and Shame - A Twitter bot which tweets stats about the BGP DFZ.
3 stars 0 forks source link

Is a max file size needed? #134

Closed jwbensley closed 1 year ago

jwbensley commented 2 years ago 
2022-06-09 08:02:18,353|INFO|Downloading http://archive.routeviews.org/route-views.linx/bgpdata/2022.06/UPDATES/updates.20220608.1815.bz2 to /opt/dnas_data/downloads/LINX/updates.20220608.1815.bz2
2022-06-09 08:02:18,650|INFO|File size is 213.4797MBs
2022-06-09 08:02:35,778|INFO|Downloaded 361/1056

Even though files above a certain threshold are parsed on a single core to stop a Python RAM apocalypse, do we need a max file size even for a single core?

jwbensley commented 1 year ago

Closing for now as this doesn't seem to be causing any problems.