Firefox nightly builds are no longer getting processed due to TC builds

[whimboo]

There was a change for RelEng lately that Nightly builds are now generated via Taskcluster. Since this switch has been turned on we no longer see the Pulse notifications coming from Buildbot. As result the Firefox UI jobs are no longer run against Nightly builds for Linux and Linux64.

CC'ing some Releng folks who could give some clarifications on what we would have to listen now. @ccooper, @Callek.

[whimboo]

Here are some examples for Linux64: https://tools.taskcluster.net/task-inspector/#eT0CZ8mYSguJbE53j3D6CQ/ (en-US) https://tools.taskcluster.net/task-inspector/#Epit-uxDQW25lkGGeoOO6w/ (ar, ast, cs, de...)

Routes we might want to use are: index.gecko.v2.mozilla-central.nightly.latest.%application%.%platform%-opt (en-US) index.gecko.v2.mozilla-central.latest.%application%-l10n.%platform%-opt.%locale% (locales)

Sadly the l10n builds don't have a nightly section in their routing keys.

[whimboo]

An example push from today with all the jobs/tests:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=5a4412474c63e1d9e66036d603ac42e9cb2b9150&filter-searchStr=nightly

[Callek]

@whimboo - First question, which revolves around a difference between BB and TC nightlies. -- Do your UI tests need to be based on Signed builds, or are unsigned ok?

index.gecko.v2.mozilla-central.nightly.latest.%application%.%platform%-opt (en-US) index.gecko.v2.mozilla-central.latest.%application%-l10n.%platform%-opt.%locale% (locales)

I should note, that yes, nightlies in the l10n space have no nightly as part of the key, but unless you're staring at try there shouldn't be (for now!) any non-nightlies in that space.

The additional complication is that the l10n routes we have now, are also multiple locales in one taskID, so you have to expand the locale you want from the index.

I also note Bug 1301495 fixed some of the routing, I have plans to add nightly routes for l10n, and there is Bug 1332407 for en-US and such in the locale space too.

[whimboo]

Our Fx ui tests via mozmill-ci are usually testing what we serve to our users. I assume we do not let them run unsigned builds. This is something we have to keep in mind when we want to integrate them into the Nightly task graph.

Multi-locales are no problem. This is something we already handle for funsize tasks in running our update tests. So code for that already exists but might have to be updated a bit to the specific data structure.

Do you have an ETA when the nightly part will be available for l10n nightly builds? If it takes too long I might want to start with the current routes, and update later.

[whimboo]

I wonder how difficult it would be to directly it would be to directly have our functional tests run via the TC graph, given that we have those actively running per checkin on all branches already.

[Callek]

@whimboo - Sadly no current ETA with the nightly specific routes. Either way I do plan to have a grace period where both end up pointing at the same thing.

I'm probably going to depend that work on Bug 1333255 Which came out of a bustage in Bug 1333234

The idea about signing, normal CI won't be signed, and currently the only plans for signed routes is to take the existing routes and append .signing to them, but that too may depend on the cited work, and has no current ETA.

My current thinking for nightly l10n routes is basically:

   "{index}.gecko.v2.{project}.nightly.{year}.{month}.{day}.revision.{head_rev}.{build_product}-l10n.{build_name}-{build_type}",
   "{index}.gecko.v2.{project}.nightly.{year}.{month}.{day}.latest.{build_product}-l10n.{build_name}-{build_type}",
   "{index}.gecko.v2.{project}.nightly.revision.{head_rev}.{build_product}-l10n.{build_name}-{build_type}",
   "{index}.gecko.v2.{project}.nightly.latest.{build_product}-l10n.{build_name}-{build_type}"