Proposal for allowing `@rust-log-analyzer silence` to mute failures

[tgross35]

It would be nice to be able to make RLA a bit less annoying when you're actively watching your PR and know it is going to fail.

Proposed usage:

silence all announcements:
@rust-log-analyzer silence silence

silence a announcements about a single test:
@rust-log-analyzer silence  mingw-check

silence multiple:
@rust-log-analyzer silence  mingw-check, mingw-test-tidy

unsilence takes the same patterns as above:
@rust-log-analyzer silence

It seems like RLA is currently stateless. Doing this would require saving some information, but changes to this would be infrequent enough that a sqlite database could handle it easily;

CREATE TABLE silenced (
    pr_id INTEGER PRIMARY KEY,
    silence_all INTEGER NOT NULL DEFAULT FALSE,
    -- JSON array of specific silenced tests
    silenced TEXT
);

Behavior:

• Accept the issue_comment webhook event.
  • If it contains silence but nothing else, upsert silence_all for that PR's ID
  • If it contains silence and some specific tests, update silence_all=false and those add those tests to a list in silenced
  • If it contains unsilence, do the opposite and delete the row if empty
• Upon writing a comment, compare the failed test to the silencing rules and don't publish it if that test is silenced

I don't mind doing the work for this, but would appreciate some feedback before starting to make sure this sounds OK.

[Mark-Simulacrum]

There's no trivial place to save the SQLite DB -- RLA runs on ephemeral containers. We do have some state (the trained index), which is persisted and loaded from S3: https://github.com/rust-lang/rust-log-analyzer/pull/71

It seems plausible that we could put a DB into S3 as well, but I wonder if perhaps a simpler design is to (for example) add a label to PRs "rla-silenced" or something like that? That would mean a binary silence rather than something more complex, but that seems like it owuld be sufficient for 99% of use cases anyway?

[tgross35]

A label sounds very reasonable to me, and easier.

Is the only thing needed a check like this one? Query the PR labels then just return this function if that one is set https://github.com/rust-lang/rust-log-analyzer/blob/64379275447650402a7a1c02901e5c4b18446b43/src/bin/server/worker.rs#L267-L273