Put more info about tests on Phabricator

[marco-c]

Right now we're only reporting the pass/fail status, we could instead report a few more data about them: https://phabricator.services.mozilla.com/conduit/method/harbormaster.sendmessage/.

[marco-c]

I'm not sure how far we want to go here, we might prefer just relying on Treeherder for details.

[marco-c]

@armenzg @camd for build failures, we are adding a bit of detail to the failures. See for example the part which reads:

Code review bot found a build error:
use of undeclared identifier 'a'; did you mean 'nsGkAtoms::a'?

on https://phabricator.services.mozilla.com/D55973.

We want to do the same for test failures, where we currently don't show any detail (e.g. https://phabricator-dev.allizom.org/D1609).

Do you have a preference between: 1) showing the failure summary; 2) showing a link to the job on Treeherder.

If (1), is there a Treeherder API to retrieve the failures summary given a task? If (2), how do I build the URL for Treeherder? (in particular, the selectedJob=XXX part)

[armenzg]

camd knows more about failure summary. I will let him answer.

For (2), I just wrote this line:

${thUrl}/#/jobs?repo=${project}&revision=${revision}&selectedJob=${job[1]}&searchStr=${job[4]}

I add searchStr for the purpose of showing less jobs on the push.

[marco-c]

@armenzg how do I know the job ID for a given task? I'm listening for task-completed, so I have the following information at my disposal:

{
  "status": {
    "taskId": "e6qjT3SdQVKQKDrsVfEi3Q",
    "provisionerId": "gecko-t",
    "workerType": "t-win10-64",
    "schedulerId": "gecko-level-1",
    "taskGroupId": "eVn6tZ7WSOCIXjS7zBe4kA",
    "deadline": "2019-12-21T12:28:23.120Z",
    "expires": "2020-01-03T12:28:23.120Z",
    "retriesLeft": 5,
    "state": "completed",
    "runs": [
      {
        "runId": 0,
        "state": "completed",
        "reasonCreated": "scheduled",
        "reasonResolved": "completed",
        "workerGroup": "aws",
        "workerId": "i-08ec1e13329698bfd",
        "takenUntil": "2019-12-20T14:35:35.026Z",
        "scheduled": "2019-12-20T13:50:14.903Z",
        "started": "2019-12-20T13:58:34.752Z",
        "resolved": "2019-12-20T14:22:58.067Z"
      }
    ]
  },
  "runId": 0,
  "task": {
    "tags": {
      "kind": "test",
      "os": "windows",
      "createdForUser": "afarre@mozilla.com",
      "retrigger": "true",
      "label": "test-windows10-64-qr/opt-mochitest-browser-chrome-fis-e10s-1",
      "test-type": "mochitest",
      "worker-implementation": "generic-worker"
    }
  },
  "workerGroup": "aws",
  "workerId": "i-08ec1e13329698bfd",
  "version": 1
}

[armenzg]

I did this (based on code from Treeherder):

>>> import slugid
>>> slugid.decode('e6qjT3SdQVKQKDrsVfEi3Q')
UUID('7baaa34f-749d-4152-9028-3aec55f122dd')

https://treeherder.mozilla.org/api/jobdetail/?job_guid=7baaa34f-749d-4152-9028-3aec55f122dd/0

It is not the perfect API but it gets you that info.

[marco-c]

The API for the failure summary would be https://treeherder.mozilla.org/api/project/mozilla-central/jobs/283993706/bug_suggestions/.