mozilla / probe-scraper

Scrape and publish Telemetry probe data from Firefox
https://mozilla.github.io/probe-scraper/
Mozilla Public License 2.0
21 stars 53 forks source link

Telemetry scraping will fail when mozilla-central moves from hg to git #680

Open chutten opened 8 months ago

chutten commented 8 months ago

mozilla-central is moving from hg to git sometime in the medium term. When that happens, we can expect hgweb to either stop or change, which will interfere with the Telemetry parts of probe-scraper (the pieces that fuel the automated intake of new or changed histograms, scalars, and events which in turn fuel tooling and the pipeline itself).

There are a couple things I can think of that we can do about this 1) Nothing. Freeze the intake of new/changed Telemetry histograms, scalars, and events. Tools like probe dictionary will stop being updated, and columns will not be added for new probes without manual schema changes. This should probably only be the chosen solution if Firefox Desktop development decides to deprecate the use of Telemetry probes for instrumentation, or otherwise can be relied upon to keep the volume of such additions low enough that manual schema editing is acceptable. 2) Migrate the scraper to become a git scraper. The crunchier parts of the code are mozparsers, and they don't care what the VCS is: the only parts to move are the bits that understand revisions, releases, and time (ohno.jpg). Shouldn't be too taxing, but as we're hoping to minimize the number of supported data collection systems in Mozilla projects going forward, it'd be likely that the migrated code wouldn't be useful for long.

No doubt there are other options.

As there is as-yet no specific date of hgweb's freezing or demise (or certainty of its fate), we can backburner this and use it more as a place to accumulate information and ideas until we hear more.