m-lab / traceroute-caller

A sidecar service which runs traceroute after a connection closes
Apache License 2.0
18 stars 5 forks source link

mlab3-lax03 has tons of traceroute tests w/ 0 probec #35

Closed yachang closed 4 years ago

yachang commented 4 years ago

https://pantheon.corp.google.com/storage/browser/_details/archive-mlab-oti/ndt/traceroute/2019/11/01/20191101T020014.044139Z-traceroute-mlab3-lax03-ndt.tgz?project=mlab-oti&organizationId=433637338589

All tests in this tarball is <1k bytes.

A typical one is like:

======================

{"UUID":"ndt-9fw2l_1572433802_000000000001C6A1","TracerouteCallerVersion":"bc092be","CachedResult":true,"CachedUUID":"ndt-9fw2l_1572433802_000000000001C6A3"} {"type":"cycle-start", "list_name":"/tmp/scamperctrl:16626", "id":1, "hostname":"ndt-9fw2l", "start_time":1572739620} {"type":"tracelb", "version":"0.1", "userid":0, "method":"icmp-echo", "src":"::ffff:173.205.3.101", "dst":"::ffff:68.190.243.205", "start":{"sec":1572739620, "usec":959242, "ftime":"2019-11-03 00:07:00"}, "probe_size":60, "firsthop":1, "attempts":3, "confidence":95, "tos":0, "gaplimit":3, "wait_timeout":5, "wait_probe":250, "probec":0, "probec_max":3000, "nodec":0, "linkc":0} {"type":"cycle-stop", "list_name":"/tmp/scamperctrl:16626", "id":1, "hostname":"ndt-9fw2l", "stop_time":1572739620}

yachang commented 4 years ago

This problem seems fixed by https://github.com/m-lab/k8s-support/pull/327

With docker image https://github.com/m-lab/traceroute-caller/releases/tag/v0.3.3

yachang commented 4 years ago

Problem fixed by

https://github.com/m-lab/traceroute-caller/releases/tag/v0.6.0

m-lab/k8s-support#347

yachang commented 4 years ago

Screenshot from 2020-01-08 09-57-28

yachang commented 4 years ago

% of tests with hops (file size > 1K) increased significantly after prod deployment.