search

mozilla / telemetry-dashboard

Web-frontend for aggregated telemetry data
telemetry.mozilla.org
Other
91 stars 114 forks source link

Broken dashboard entry "SCALARS_49" etc. #382

Open georgf opened 6 years ago

georgf commented 6 years ago

I see some odd entries of the form "SCALARS_" like SCALARS_49. Trying to select them results in errors like:

uncaught exception: Could not obtain evolution: status 500 (https://aggregates.telemetry.mozilla.org/aggregates_by/build_id/channels/beta/?version=58&dates=20180118%2C20180117%2C20180116%2C20180115%2C20180113%2C20180112%2C20180111%2C20180110%2C20180109%2C20180108%2C20180105%2C20180104%2C20180103%2C20171228%2C20171227%2C20171226%2C20171223%2C20171222%2C20171221%2C20171220%2C20171219%2C20171218%2C20171216%2C20171215%2C20171212%2C20171211%2C20171210%2C20171209%2C20171208%2C20171207%2C20171206%2C20171205%2C20171204%2C20171203%2C20171202%2C20171201%2C20171130%2C20171129%2C20171128%2C20171127%2C20171125%2C20171124%2C20171123%2C20171122%2C20171121%2C20171120%2C20171119%2C20171117%2C20171116%2C20171115%2C20171114%2C20171113%2C20171112%2C20171111%2C20171110%2C20171109%2C20171108%2C20171107%2C20171106%2C20171104%2C20171103%2C20171102%2C20171030%2C20171024%2C20171023%2C20171018%2C20171017%2C20171015%2C20171013%2C20171008%2C20171006%2C20171002%2C20170930%2C20170928%2C20170927%2C20170926%2C20170924&metric=SCALARS_49&application=Firefox)

TypeError: gCurrentHistogramsList is null[Learn More]

georgf commented 6 years ago

More noise:

chutten commented 6 years ago

@fbertsch Are those actually in the db?

fbertsch commented 6 years ago

Yes, they are, unfortunately. What happened is this: Histograms all of their buckets for the db, so we use the Histogram class to retrieve it [0]. We don't need to do this for scalars or simple_measures, since they already contain the entirety of their information.

Using the Histogram class has the added benefit - or detrimental effect - of only including histograms that are in the current histograms.json! So that's why we never see these errors with histograms.

The actual error comes from the service, when it tries to finally load a definition for these scalars, and it finds none: moztelemetry.scalar:MissingScalarError: 'Definition not found for 49'.

What should happen is the following (as work items):

  1. Catch the MissingScalarError and 404 that data
  2. Check for definitions of these scalars before inserting them into the DB
  3. Check for definitions of simple measures, when we use the probe info service

HOWEVER, take note that this means that historical histograms are not aggregated (news to me)! If we make change (2.) before switching to getting historical scalars from the probe info service, the same will be true of scalars.

[0] https://github.com/mozilla/python_mozaggregator/blob/master/mozaggregator/db.py#L89