search

m-lab / etl

M-Lab ingestion pipeline
Apache License 2.0
22 stars 7 forks source link

Add hopannotation1 to paris traceroute #1047

Closed cristinaleonr closed 2 years ago

cristinaleonr commented 2 years ago

Notes regarding hop annotations for paris traceroute:

We're using the LogTime as the timestamp in the ID and for the Timestamp field itself. This is the only timestamp that I see in the .paris files and it comes from the filename. I'm not sure if this is equivalent to the CycleStartTime we adopted as a convention for hopannotation1's timestamp.

Example hop IP, hostname, and ID: Screenshot 2022-02-02 7 51 31 AM

This change is Reviewable

coveralls commented 2 years ago

Pull Request Test Coverage Report for Build 7149

Totals Coverage Status
Change from base Build 7147: 0.1%
Covered Lines: 3855
Relevant Lines: 5997
💛 - Coveralls
SaiedKazemi commented 2 years ago

parser/pt.go, line 425 at r2 (raw file):

Previously, stephen-soltesz (Stephen Soltesz) wrote…
Are these the correct three fields composing the hop id from traceroute-caller? https://docs.google.com/document/d/1VD5t4zRFlYsq_CAH6oCfjynKaEDnpwHnnFbk2mL7_yI/edit?resourcekey=0-NpaXaG91iS8nb02o9fqgbQ#heading=h.mzu1y33z63fh Also, FYI: @saiedkazemi (I don't want to miss anything)

Looks good to me. Here's what the doc says:

The hop annotation file name should consist of the following three fields concatenated with an underscore character (_) and followed by a .json suffix:

  1. Timestamp - This should be the timestamp when traceroute ran (see Hop Annotation File Content for more details).
  2. Hostname - This should be the hostname where TRC is running on.
  3. IP address - This should be the IP address of the hop that is annotated.
SaiedKazemi commented 2 years ago

parser/pt.go, line 425 at r2 (raw file):

Previously, stephen-soltesz (Stephen Soltesz) wrote…
@saiedkazemi should the `Hostname` be that of the M-Lab server or the reverse-lookup hostname of the hop router? Which is being used here?

@stephen-soltesz Per our offline conversation, the hostname in the screenshot ge-11-0-2... is not what we want. I will set up a meeting with @cristinaleonr to discuss this.

stephen-soltesz commented 2 years ago

Part of https://github.com/m-lab/etl/issues/1050