mozilla / glam

Mozilla's primary interactive dashboard for examining the distribution of telemetry values.
https://glam.telemetry.mozilla.org
Mozilla Public License 2.0
20 stars 23 forks source link

Can we apply events_stream performace improvements to GLAM ETL? #2840

Closed edugfilho closed 5 days ago

edugfilho commented 1 month ago

Following @BenWu 's improvement to events_stream's queries in https://github.com/mozilla/bigquery-etl/pull/5659 we should see if we can:

BenWu commented 1 month ago

Some notes re:js udfs, it would be good to see the maximum improvement we can get from replacing the js udfs before committing too much time into it. One quick way is to just replace the udfs with stub udfs that do very little and see how much slot time reduces.

From a quick look at mozfun.glam, the js udfs are:

Looking at where these are used, I wouldn't expect a huge improvement from replacing these but I think it's low-effort enough to try.

I created a sample pr with the three of the fairly trivial conversions at https://github.com/mozilla/bigquery-etl/pull/5689