search

m-lab / ndt

Network Diagnostic Tool
Other
11 stars 7 forks source link

Missing .meta files for about 10% of tests. #91

Open gfr10598 opened 7 years ago

gfr10598 commented 7 years ago

It appears that for about 10% of the NDT tests in recent months, about 10% of the tests have s2c or c2s files, but no corresponding .meta file. We have verified by looking at the files directly on mlab servers that the meta files are already missing there. It is also clear that they are missing immediately, and not somehow being deleted from the file system later.

gfr10598 commented 7 years ago

We have better stats on this now. OneBox collects download as one test group, then upload as a separate test group. This may be a factor.

For a typical day's day, we see about 800K tests altogether. Among these we see: 293K s2c with meta 65K s2c tests that have no meta file, AND are uncompressed.

279K c2s with meta 21K c2s tests that have no meta file, AND are uncompressed.

This suggests that the downloads are being interrupted something like 15% of the time, such that the file doesn't get compressed and the meta file never gets written. The uploads are being interrupted something like 7% of the time.