agardnerIT / tracepusher

Generate and push OpenTelemetry Trace data to an OTEL collector in JSON format
Apache License 2.0
54 stars 11 forks source link

har-to-otel - start-time was in an incorrect format #93

Closed eravion closed 2 months ago

eravion commented 3 months ago

Hello,

Testing the import of HAR file to jaeger or grafana tempo, but the date of the record is not the one of the HAR.

In order to reproduce, the output of har-to-tel, that I'll use with tracepusher.py to test :

ARGS TO TRACEPUSHER: --endpoint http://X.X.X.X:4318 --insecure true --service-name har-to-otel --span-name 'https://images-na.ssl-images-amazon.com/images/G/01/AUIClients/ClientSideMetricsAUIJavascript-51171fbdd28e1a7a61e922e8f0272af8bc74d37b.secure.variant-desktop-session-snapshot-keypress.min._V2_.js' --duration 18 --duration-type 'ms' --trace-id 85fe5e274d38fb5e12c3ce2515593160 --span-id f5d0dc3756eb5ebe --start-time '**2024-07-30T05:00:35.885+0200**' --span-attributes serverIPAddress='2.16.238.142' request.method=GET request.httpVersion=h2 request.headers.sec-ch-ua-platform="Linux" request.headers.sec-ch-ua="Not_A Brand;v=8, Chromium;v=120, HeadlessChrome;v=120" request.headers.Referer="https://www.amazon.com/" request.headers.sec-ch-ua-mobile="?0" request.headers.User-Agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/120.0.6099.224 Safari/537.36" request.headers.sec-fetch-mode="no-cors" request.headers.:scheme="https" request.headers.referer="https://www.amazon.com/" request.headers.sec-ch-ua-mobile="?0" request.headers.user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/120.0.6099.224 Safari/537.36" request.headers.:method="GET" request.headers.:path="/images/G/01/AUIClients/ClientSideMetricsAUIJavascript-51171fbdd28e1a7a61e922e8f0272af8bc74d37b.secure.variant-desktop-session-snapshot-keypress.min._V2_.js" request.headers.accept="*/*" request.headers.sec-fetch-dest="script" request.headers.:authority="images-na.ssl-images-amazon.com" request.headers.accept-encoding="gzip, deflate, br" request.headers.sec-ch-ua-platform="Linux" request.headers.sec-ch-ua="Not_A Brand;v=8, Chromium;v=120, HeadlessChrome;v=120" request.headers.sec-fetch-site="cross-site" response.status=200 response.statusText="" response.httpVersion=h2 response.content.size=19614 response.headers.cache-control="public, max-age=627132845" response.headers.content-encoding="br" response.headers.server="Server" response.headers.server-timing="provider;desc=ak" response.headers.timing-allow-origin="https://www.amazon.com" response.headers.x-cache="Hit from akamai" response.headers.akamai-cache-status="Hit from child" response.headers.date="Tue, 30 Jul 2024 03:00:35 GMT" response.headers.expires="Mon, 13 Jun 2044 14:34:40 GMT" response.headers.peer-cache="Hit" response.headers.vary="Accept-Encoding" response.headers.access-control-allow-origin="*" response.headers.alt-svc="h3=:443; ma=93600" response.headers.x-nginx-cache-status="HIT" response.headers.akamai-grn="0.8eed1002.1722308435.5f2b061b" response.headers.content-length="7642" response.headers.content-type="application/x-javascript" response.headers.last-modified="Thu, 01 Oct 2015 10:17:43 GMT" response.headers.surrogate-key="x-cache-012 /images/G/01/AUIClients/ClientSideMetricsAUIJavascript-51171fbdd28e1a7a61e922e8f0272af8bc74d37b.secure.variant-desktop-s" response.headers.x-amz-ir-id="13c87f43-e1e1-41d5-9f1f-89901aea496a" response.headers.RemoteAddress="2.16.238.142" response.headers.X-GeoIP-Country="DE" response.headers.X-GeoIP-RegionName="Hesse" response.headers.X-GeoIP-Latitude="50.1155" response.headers.X-GeoIP-RegionCode="60306" response.headers.X-GeoIP-City="Frankfurt am Main" response.headers.X-GeoIP-Longitude="8.6842" response.headers.X-GeoIP-AsCode="20940" response.headers.X-GeoIP-AsLabel="Akamai International B.V." timings.blocked=0.233 timings.dns=-1 timings.connect=-1 timings.send=0.089 timings.wait=16.309 timings.receive=1.027 timings.ssl=-1 --debug False --dry-run False

startedDateTime for this line in HAR file is 2024-07-30T05:00:35.885+0200

Testing with tracepusher.py

$ python3 ./tracepusher.py --endpoint http://X.X.X.X:4318 --insecure true --service-name har-to-otel --span-name 'https://images-na.ssl-images-amazon.com/images/G/01/AUIClients/ClientSideMetricsAUIJavascript-51171fbdd28e1a7a61e922e8f0272af8bc74d37b.secure.variant-desktop-session-snapshot-keypress.min._V2_.js' --duration 18 --duration-type 'ms' --trace-id 85fe5e274d38fb5e12c3ce2515593160 --span-id f5d0dc3756eb5ebe --start-time '2024-07-30T05:00:35.885+0200' --span-attributes serverIPAddress='2.16.238.142' request.method=GET request.httpVersion=h2 request.headers.sec-ch-ua-platform="Linux" request.headers.sec-ch-ua="Not_A Brand;v=8, Chromium;v=120, HeadlessChrome;v=120" request.headers.Referer="https://www.amazon.com/" request.headers.sec-ch-ua-mobile="?0" request.headers.User-Agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/120.0.6099.224 Safari/537.36" request.headers.sec-fetch-mode="no-cors" request.headers.:scheme="https" request.headers.referer="https://www.amazon.com/" request.headers.sec-ch-ua-mobile="?0" request.headers.user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/120.0.6099.224 Safari/537.36" request.headers.:method="GET" request.headers.:path="/images/G/01/AUIClients/ClientSideMetricsAUIJavascript-51171fbdd28e1a7a61e922e8f0272af8bc74d37b.secure.variant-desktop-session-snapshot-keypress.min._V2_.js" request.headers.accept="*/*" request.headers.sec-fetch-dest="script" request.headers.:authority="images-na.ssl-images-amazon.com" request.headers.accept-encoding="gzip, deflate, br" request.headers.sec-ch-ua-platform="Linux" request.headers.sec-ch-ua="Not_A Brand;v=8, Chromium;v=120, HeadlessChrome;v=120" request.headers.sec-fetch-site="cross-site" response.status=200 response.statusText="" response.httpVersion=h2 response.content.size=19614 response.headers.cache-control="public, max-age=627132845" response.headers.content-encoding="br" response.headers.server="Server" response.headers.server-timing="provider;desc=ak" response.headers.timing-allow-origin="https://www.amazon.com" response.headers.x-cache="Hit from akamai" response.headers.akamai-cache-status="Hit from child" response.headers.date="Tue, 30 Jul 2024 03:00:35 GMT" response.headers.expires="Mon, 13 Jun 2044 14:34:40 GMT" response.headers.peer-cache="Hit" response.headers.vary="Accept-Encoding" response.headers.access-control-allow-origin="*" response.headers.alt-svc="h3=:443; ma=93600" response.headers.x-nginx-cache-status="HIT" response.headers.akamai-grn="0.8eed1002.1722308435.5f2b061b" response.headers.content-length="7642" response.headers.content-type="application/x-javascript" response.headers.last-modified="Thu, 01 Oct 2015 10:17:43 GMT" response.headers.surrogate-key="x-cache-012 /images/G/01/AUIClients/ClientSideMetricsAUIJavascript-51171fbdd28e1a7a61e922e8f0272af8bc74d37b.secure.variant-desktop-s" response.headers.x-amz-ir-id="13c87f43-e1e1-41d5-9f1f-89901aea496a" response.headers.RemoteAddress="2.16.238.142" response.headers.X-GeoIP-Country="DE" response.headers.X-GeoIP-RegionName="Hesse" response.headers.X-GeoIP-Latitude="50.1155" response.headers.X-GeoIP-RegionCode="60306" response.headers.X-GeoIP-City="Frankfurt am Main" response.headers.X-GeoIP-Longitude="8.6842" response.headers.X-GeoIP-AsCode="20940" response.headers.X-GeoIP-AsLabel="Akamai International B.V." timings.blocked=0.233 timings.dns=-1 timings.connect=-1 timings.send=0.089 timings.wait=16.309 timings.receive=1.027 timings.ssl=-1  --debug False --dry-run False
Got an explicit start time
**WARN: --start-time was in an incorrect format. If providing a date/time it must be in UTC and end with uppercase 'Z'. eg. '2023-11-26T03:05:16.844Z'. Trace will be sent with start_time of now.**
<Response [200]>

Note : the debug mode is very helpful, thanks for that :)

agardnerIT commented 2 months ago

The ONLY input formats tracepusher currently supports are: 1) A 19 digit string representing milliseconds since the epoch: eg. 1700967916494000000 2) "%Y-%m-%dT%H:%M:%S.%fZ" eg. "2023-11-26T03:05:16.844Z"

https://github.com/agardnerIT/tracepusher/blob/main/tracepusher.py#L292

This is an area that I'd be happy for support and to review PRs if you'd be willing to contribute?