Possible to send build numbers / tagging info to Sauce Labs for annotation purposes?

[jzombie]

Is it possible to add build numbers, and / or tagging, to Sauce Labs builds by using this?

https://wiki.saucelabs.com/display/DOCS/Annotating+Tests+with+the+Sauce+Labs+REST+API

My builds show up as Sauce Labs Unknown in the badge status. The simple-peer repository (https://github.com/feross/simple-peer) doesn't seem to be annotating with build numbers, and their badge shows up as building just fine.

[vweevers]

How are build numbers and badges related? We do send a build number and name to Sauce Labs:

https://github.com/airtap/sauce/blob/a18342709ae5be520a65610a84184b873ab59eeb/index.js#L136-L137

Could it be a cache issue (which is common with badges)? How long ago did your build run?

[jzombie]

I last ran it 10 hours ago: https://app.saucelabs.com/open_sauce/user/zenosmosis/tests/vdc

My badge URL is https://saucelabs.com/buildstatus/zenosmosis; repo is https://github.com/jzombie/webrtc-peer

[vweevers]

Try using the https://app.saucelabs.com/ domain, I saw on matrix images that the saucelabs.com no longer works.

[jzombie]

Same results: https://app.saucelabs.com/buildstatus/zenosmosis

[jzombie]

In regards to the build correlation, this is the email I received from one of their engineers.

---

Our engineers mentioned that the badge shows unknown, because you're not using builds. Once you add a build to your tests, the badge will work.

For example, this test shows null for the build value. https://app.saucelabs.com/tests/e3a04d4372854163a897e1a0ee79499e

https://wiki.saucelabs.com/display/DOCS/Best+Practice%3A+Use+Build+IDs%2C+Tags%2C+and+Names+to+Identify+Your+Tests

Best, Hanssen

[vweevers]

I tried a bunch of others (abstract-leveldown, level-js, level-ci - all using Airtap 4) and their badges work. Maybe that's because they previously used Airtap 3 and the badge status still reflects old builds.

I'll check the build numbers.

[jzombie]

Thanks!

[vweevers]

I reckon it's because https://github.com/nkt/build-number/blob/master/index.js doesn't support GitHub Actions. We can add a github-specific id here:

https://github.com/airtap/sauce/blob/a18342709ae5be520a65610a84184b873ab59eeb/index.js#L132

I.e. const build = buildNumber() || process.env.GITHUB_RUN_ID.

[jzombie]

Nice, I can do a temporary fork and try it if needed.

[vweevers]

Please do, thank you!

I'm not 100% sure GITHUB_RUN_ID is the right variable. It's documented as:

A unique number for each run within a repository. This number does not change if you re-run the workflow run.

Which, if Sauce Labs supports this, should mean that a previous build status will be overwritten (e.g. when a temporary failure occurred and you retry the workflow run).

[jzombie]

There is also GITHUB_RUN_NUMBER.

If I get it to work, I'll submit a patch.

Thanks for the help getting me started here.

[jzombie]

Here's the PR: https://github.com/airtap/sauce/pull/9