Support Telemetry

[Yoric]

(probably a subset of #241)

To perform real-world performance testing, Telemetry would be extremely useful.

[SimonSapin]

Yes, I’d like to have data about #6564 for example.

Do you have suggestions of how to approach this? I mean on the high level design. Could we share some of Firefox’s infrastructure?

[Yoric]

Well, Telemetry in Firefox is C++ and .jsm, plus the server back-end. I believe that the server back-end could be reused immediately (it works for several products already). The client side, however, well, that depends on the policy of shipping C++/jsm code in Servo.

[Yoric]

Cc @georgf.

To clarify, Telemetry is:

1. a set of histograms (name, description, histogram kind, value range), used both on the client (to update the histograms) and on the server (to display the description, labels, etc.);
2. a client-side API used to add values to histograms or to read values from histograms;
3. a client-side page about:telemetry which uses the above API to display histograms;
4. some client-side back-end to persist the values to disk and upload them;
5. lots of server infrastructure.

Since the protocol is documented, I believe that Servo can reuse the server infrastructure without any change. I suspect that all of the rest would need to be reimplemented.

[georgf]

CC @vdjeric, @Dexterp37.

We have documentation of Telemetry here: https://gecko.readthedocs.org/en/latest/toolkit/components/telemetry/telemetry/index.html https://wiki.mozilla.org/Telemetry

I don't know how feasible it is, but ideally we would re-use as much of the same code as possible (we already spent a lot of time on getting e.g. submission & persistence right). The minimal overlap should be using the same histogram definitions and top-level ping format, as that gives you easy server-side and dashboard support.

[jdm]

It is possible for us to call into C++ APIs using @michaelwu's modified rust-bindgen.

[jdm]

Orrrrrrr we could rewrite the telemetry implementation in Rust >:D

[Yoric]

@georgf How much of Telemetry is written in C++ vs JS?

[vdjeric]

$ ls *.jsm | xargs wc -l | grep total 8193 total

$ ls .h .cpp | xargs wc -l | grep total 4855 total

[jdm]

Oof, the amout of JSMs present in http://mxr.mozilla.org/mozilla-central/source/toolkit/components/telemetry/ doesn't fill me with optimism. We've got no ability to execute JS outside of web content right now.

[georgf]

The other option is to do what B2G does right now - re-use the C++ parts, nsITelemetry et al, to get histogram collections and serialization, then implement ping generation and submission yourself: https://github.com/mozilla-b2g/gaia/blob/master/shared/js/telemetry.js

The JSMs are implementing ping generation, persistence, submission and desktop-specific measurements.

[Yoric]

I wouldn't hold my breath on actually reusing the jsm, even with ability to execute priviledged JS content, as it depends on many libraries. However, I believe that the algorithms could be ported. It's work, but not impossible.

[vmunix]

+1 to a full rust re-implementation of at least ping generation and submission, which ought to be easy, right? :) (says the guy who doesn't have any skin in the game..)

Maybe @trink needs a weekend Rust project? :smile:

[Yoric]

Perhaps it would be interesting to implement Telemetry as an independent library. If I find time, I'll take a look at how difficult this would be.

[Yoric]

I have posted an early design here. For the moment, I have focused only on the signature dealing with creating/updating histograms. The objective would be to have the share with Gecko Telemetry the on-disk format, the upload protocol and the algorithms, as well as the server code, but probably not the client code.

Comments appreciated (in particular @georgf , @Dexterp37, @SimonSapin ).

Edit Updated URL to point to the generated documentation.

[Yoric]

After discussing with @georgf , we agreed that a Telemetry library should not handle upload or storage to disk, as both are certainly application-specific. So, I have written an advanced prototype of the core of Telemetry, which handles only the following things:

• creating histograms;
• activating/deactivating them by "feature groups" (as typically, some histograms will be privacy-sensitive);
• updating histograms;
• serializing the data.

Source is here, generated documentation is here. Once I am satisfied with the library, I will publish it on cargo, so as to be able to test it with Servo.

[jdm]

I think a simple example for both uploading and persisting to disk would be useful additions to an examples/ directory.

[Yoric]

@jdm filed.

[Yoric]

In its current state (I'm basically ready to release 0.1.0), Telemetry.rs is sufficient to capture data and store it to disk/dump it as needed. I believe that this is already a useful state. I have added a full example with multiple threads and disk storage.

The library is not ready to upload to a server, as I have not reverse-engineered the protocol we use for communicating with the server. The missing bit is the serialization format for histograms, which is entirely unspecified.

Once we have both, we'll still need to work on providing the data surrounding histograms and on accessing the REST API (I haven't looked for the doc yet).

[georgf]

I don't think you need to cover the full main ping format, but this should comply to the common ping format (without environment & clientId probably). You could just push something like { histograms: {...} } as the payload and be done. Maybe it might make sense to line up closer with the Firefox desktop format later for comparison, but i assume thats not a concern now.

The API to push to the server is in the Edge server specification.

If you happen to come up with notes or a short writeup for the current histogram format, i'd love to integrate that in our mozilla-central docs :)

vdjeric or rvitillo probably know best how to get that data into the dashboard aggregates from there, but given a compatible histogram format that should be not too crazy.

[Yoric]

Thanks for the push API and good to know that we don't need the environment and clientId, that will simplify things. Filed reverse-engineering the current histogram format. I'll work on that for 0.2.

[Yoric]

@georgf , the Edge API requires a reason (in addition to the Telemetry Ping field reason). Which should we provide? shutdown?

[georgf]

You can use the ping "type", i think that page is just not updated to the new terminalogy. The new ping submission path for Firefox just uses the type FWIW.

[Yoric]

See above for an early draft of Telemetry in Servo. Feedback welcome.

[jdm]

Do the restrictions become easier if we use lazy_static and store them in an Arc like this?

[jdm]

Ah, I see you address that in the RFC.

[Yoric]

Updated draft, with a demo of how to add a histogram to the compositor, without having to patch every single file. Feedback welcome.