Missing metrics for vimeo and yandex due to: AttributeError: 'list' object has no attribute 'get'

[stephendonner]

We're missing data for vimeo and yandex (in DataDog & probably Telemetry) due to the following Python traceback[0]:

23:24:17 [vimeo] Running shell script
23:24:17 + python ./send_to_telemetry.py wpt.json
Post stage
23:24:18 {'appName': 'firefox', 'channel': 'release', 'version': '65.0.1', 'connection': 'cable', 'url': 'https://vimeo.com', 'platform': 'desktop', 'runner': 'https://qa-preprod-master.fxtest.jenkins.stage.mozaws.net/', 'runId': 'jenkins-wpt-vimeo-601', 'sessionState': 'noAuth', 'metrics': {'TTFB': {'firstView': {'median': 237, 'standardDeviation': 1}}, 'render': {'firstView': {'median': 2500, 'standardDeviation': 47}}, 'firstPaint': {'firstView': {'median': 933, 'standardDeviation': 41}}, 'timeToContentfulPaint': {'firstView': {'median': 1552605715418, 'standardDeviation': 1117}}, 'domContentFlushed': {'firstView': {'median': 2505, 'standardDeviation': 24}}, 'timeToFirstInteractive': {'firstView': {'median': 1552605717379, 'standardDeviation': 1273}}, 'pageLoadTime': {'firstView': {'median': 27877, 'standardDeviation': 10886}}, 'SpeedIndex': {'firstView': {'median': 4754, 'standardDeviation': 1126}}, 'bytesInDoc': {'firstView': {'median': 12980254, 'standardDeviation': 5166800}}, 'visualComplete': {'firstView': {'median': 30000, 'standardDeviation': 12552}}, 'requestsFull': {'firstView': {'median': 403, 'standardDeviation': 28}}}}
23:24:18 Traceback (most recent call last):
23:24:18   File "./send_to_telemetry.py", line 95, in <module>
23:24:18     main(*sys.argv[1:])
23:24:18   File "./send_to_telemetry.py", line 41, in main
23:24:18     sample = test["data"][measure]["firstView"].get(name)
23:24:18 AttributeError: 'list' object has no attribute 'get'
[Pipeline] archiveArtifacts
23:24:18 Archiving artifacts

/cc @davehunt @stuartphilp just FYI - will look into but might need help

[0] https://qa-preprod-master.fxtest.jenkins.stage.mozaws.net/job/wpt/job/vimeo/601/console

[davehunt]

@stephendonner do you have the JSON file that caused this exception? the stack suggests that data.[measure].firstView is an empty list instead of a dict.

[davehunt]

Looking at https://qa-preprod-master.fxtest.jenkins.stage.mozaws.net/job/wpt/job/vimeo/624/artifact/wpt.json the data.standardDeviation.firstView for Firefox Nightly is empty. Is something failing when running this test? Can we run it manually? The other tests seem to have the expected data.

[davehunt]

We should at least catch this exception so that we continue to submit the remaining results. We might want to submit the median values for Firefox Nightly, too.

[davehunt]

I've opened https://github.com/mozilla/wpt-api/pull/269