tweaselORG / cyanoacrylate

Toolkit for large-scale automated traffic analysis of mobile apps on Android and iOS.
MIT License
5 stars 1 forks source link

Our HAR files don't load in the Chrome dev tools #45

Closed baltpeter closed 1 month ago

baltpeter commented 2 months ago

While working on https://github.com/tweaselORG/docs.tweasel.org/issues/7, I noticed that the HAR files we create (via mitmproxy's har_dump) don't load in Chrome.

Example HAR (renamed to .json, because GitHub doesn't allow .har uploads): be.stib.mivb.mobile.har.json

If I try to load that in the Chrome network tab, I get:

Failed to load HAR file with following error: Invalid date format

image

I am running: Chromium Version 123.0.6312.122 (Official Build) snap (64-bit)

baltpeter commented 2 months ago

Looks like the problem is in log.pages.

For us, this currently always looks like this:

{
   "pages": [
        {
            "pageTimings": {}
        }
    ]
}

Whereas Chrome actually fills it:

{
    "pages": [
      {
        "startedDateTime": "2024-05-14T05:58:18.075Z",
        "id": "page_1",
        "title": "https://spiegel.de/",
        "pageTimings": {
          "onContentLoad": 1034.15900003165,
          "onLoad": 1035.6270000338554
        }
      }
    ]
}

The problem appears to be the missing startedDateTime. If I manually add that, the HAR loads fine.

{
    "pages": [
        {
            "startedDateTime": "2024-05-14T05:58:18.075Z",
            "pageTimings": {}
        }
    ],
}
baltpeter commented 2 months ago

This was fixed in mitmproxy 10 (we depend on v9): https://github.com/mitmproxy/mitmproxy/pull/5814

Indeed, with just an empty pages array, it works in both Firefox and Chrome.

baltpeter commented 2 months ago

Since mitmproxy 10.1 brings native HAR export anyway, I guess it makes most sense to defer this issue to #36.

baltpeter commented 2 months ago

I've changed my mind. This is quite an unfortunate problem and I would really rather not keep producing incorrect HAR files.

While I do indeed not have the time to properly test upgrading to a new mitmproxy version right now, this is easy enough to hotfix on our end.