Proposal: gather bug-report logs with "telemetry/rage"

[ljw1004]

Motivation

Some mobile apps let you shake them to report+upload an error. Visual Studio and VSCode have a "smiley/frowney face" to click which gathers relevant logs and uploads them to a bug-reporting-center. In my company, we have a similar bug button in Nuclide which gathers logs and uploads them.

I want to hook things up so these "relevant logs" can include error reports from the LSP server.

I've implemented it so far in the Nuclide editor and the Hack LSP server. It's been in production for about two months, corresponding to about 40 bugs filed. It has increased my "can I repro the bug" rate from about 10% to 90%+. The main credit for this is because my telemetry/rage dumps include complete full transcripts of every single jsonrpc interaction with the LSP server over the past five minutes, in a rolling buffer.

• Nuclide - https://github.com/facebook/nuclide/blob/96dfc9240d2d66386d5032f48a04fee4dc5aaee6/pkg/nuclide-vscode-language-service-rpc/lib/LspLanguageService.js#L1742
• Hack - https://github.com/facebook/hhvm/blob/a87a6d300711e283c01c910b68c48b005fe02b0e/hphp/hack/src/client/clientLsp.ml#L1566

Here's my proposal

Rage Request

The rage request is sent from the client to the server, typically in response to the user clicking a "submit bug" button, to gather whatever additional logs the server wishes to provide.

Request:

• method: 'telemetry/rage'
• params: void

Response:

• result: RageItem[] as defined below.

/**
 * Represents a piece of dump/debug/log information provided by the LSP server
 */
interface RageItem {
        /**
         * An optional title for the rage item so the bug report can contain named attachments.
         * By convention, use [host:]/path/file[:extra]
         * If title is null, the item data will typically go in the body of the bug report
         */
        title: string | null;

        /**
         *The string content of this bug item
         */
        data: string;
}

Registration Options:

interface ServerCapabilities {
        ...

        /**
         * The server provides rage support
         */
        rageProvider?: boolean;
}

Discussion

We could go full-blown "mime" for the attachments. That would be useful if the telemetry logs from the LSP server needed to include binary data. So far I've never needed that, so I didn't include it. But it would probably be the right thing to do.

I've written RageItem to have an optional title, with the idea that the main body of the bug report should be formed by concatenating all the RageItems that came back with empty titles. I did this as the cheapest+laziest way to get compositionality: it means that the top-level of Nuclide can send out rage requests to all the providers -- all the projects, all the source control providers, all the LSP servers that are active; each of those in turn can send out subsidiary rage requests. I wanted lots of these components to be able to provide bug information in the main body of the bug report.

I've written out a convention for names of the attachments in the bug report. For my experiences in Nuclide+Hack, I wanted the bug reports to include things like "This is the version of the document that existed prior to the DidChange event, this is the version that existed after the DidChange event, this is how the symbol table was updated". It was useful when reviewing the bugs to have all the attachments from all the providers be in a clear canonical format.

The convention I used is [host:]/path/file[:extra]. I imagine that most LSP servers operate only upon local files so they would emit titles like /home/ljw/file.php or /home/ljw/file.php:prev. But also, once we augment VSCode to allow a remote connection to an LSP server, then that remote proxy would often insert the hostname dev1.mycompany.com:/home/ljw/file.php. I've used that because my users typically have multi-root workspaces, where one root points to a filepath on a test machine, and another root points to the same filepath on a production machine.

Something I haven't addressed here is the social aspect -- if the user clicks the bug button in their editor, and the user believes the bug is related to a particular VSCode package or a particular LSP server, then how can the bug be filed somewhere in such a way that the authors of the LSP server get to see it? I think this is purely up to the editor team to solve.

[dbaeumer]

@ljw1004 thanks a lot for the proposal. Can I motivate you to provide this as a proposal contribution like documented here: https://github.com/Microsoft/language-server-protocol/blob/master/contributing.md

[dbaeumer]

I anything is unclear feel free to ping.

[puremourning]

The main credit for this is because my telemetry/rage dumps include complete full transcripts of every single jsonrpc interaction with the LSP server over the past five minutes, in a rolling buffer.

how do you deal with the information security issue here? User's proprietary IP could be included in this log dump, no?

[dbaeumer]

I think this needs to be handle by each extension using its own telemetry channel. The extension needs to let the user know about this with an option to opt out of telemetry (as VS Code does).

[ljw1004]

cc. @kieferrm