Closed jaredlockhart closed 2 years ago
Add a comment below describing the changes at a high level, including all new/deprecated fields, their types, and their uses Link to any related proposal documentation
Related documentation is mainly this Nimbus Rollouts Technical Plan
In particular, adding a per-experiment isRollout
boolean property to the experiment DTO per the "System Design, Mega MVP 2" section"
Create a PR which introduces the schema changes
https://github.com/mozilla/nimbus-shared/pull/179
Will see if any tests are relevant here
File tickets/epics for all Nimbus clients to implement the changes Nimbus Mobile
Looks like the ticket for Mobile has already been filed and implemented
Nimbus Desktop
And filed & in-progress for desktop as well
Considering the changes recently introduced by https://github.com/mozilla/nimbus-shared/issues/182, the testing recipes might need to take into account the multifeature changes as well. For testing we would also need to use both single feature and multifeature experiments that use the new schema for Rollouts.
Looks like the ticket for QA on Desktop and Mobile is here:
Merged the schema change, but have not yet made a release. We have some sample data candidates here that I'd like to wait for before the release:
https://github.com/mozilla/nimbus-shared/pull/179#issuecomment-1012215766
But also, since we've gone back & forth so much on that data, I'd also like to wait until QA for the clients is complete so we know exactly what data was practically useful.
Current testing data based on https://github.com/mozilla/nimbus-shared/pull/179#issuecomment-1012215766 (feature config values - e.g. loadMore -> true/false - and channel may change as testing is performed. Filters that can be found on Stage may be added to change the order in which we enroll in the available recipes):
Desktop:
Firefox Android:
Firefox iOS:
Final testing data used:
Desktop:
Firefox Android:
Firefox iOS:
We have finished testing. You can find the TestRail test runs here. We have found the following issues:
Amazing thank you I'll wait for https://bugzilla.mozilla.org/show_bug.cgi?id=1752146 to be resolved and then move this forward.
After the clarification of an expected behavior, this issue was also filed: EXP-2156 NEW Mobile Nimbus rollouts are not affected by the studies toggle
Okay blocking on https://mozilla-hub.atlassian.net/browse/EXP-2117 and then we can land the Experimenter parts
Thank you @heres-maria for the thorough testing!
Merged the schema change, but have not yet made a release. We have some sample data candidates here that I'd like to wait for before the release:
@jaredlockhart trying to wade through these comments. Is this issue good to close now, or do we still need to wait? Can this be released (I'm seeing the last release was 1.7.0 back in December)?
The new version v1.8.0 is being consumed by experimenter now. Closing this 👍
Welcome Change Captain! ⛵️
When making schema changes, follow this guidance to avoid breaking changes:
Rather than modifying an existing field, consider adding a new field with a new name and later deprecating the old field when it is no longer used
When deprecating an existing field, consider making it optional/nullable rather than removing it altogether and then removing it entirely when all clients/active experiments no longer depend on it
Proposing Schema Changes
When Client Changes Are Complete
When Client QA is Complete
When Bugs Are Resolved and Clients Are Released
[ ] Comment below indicating which version of each client application includes the changes
[x] File tickets/epics in Nimbus Engineering to implement the changes in Experimenter and add them in a comment below, including:
[ ] File tickets/epics in Jetstream to adapt it to any incoming changes that affect its analysis
When Experimenter Changes Are Complete