[Intermittent] Version 1.0.4 overwrites the 1.1.0rc1 add-on when enrolled in the Normandy test recipe

[SoftVision-CarmenFat]

[Notes]:

• Please note that the steps to reproduce from this report are not always leading to this wrong behavior. You might need to restart the browser for several times in order to encounter this bug.

[Affected Platforms]:

• All Windows

[Affected Versions]:

• Firefox Release 70.0.1, Build ID 20191030021342

[Prerequisites]:

• Have an active Normandy testing recipe for DoH.
• You are enrolled in the Treatment branch.

[Steps to reproduce]:

1. Open the browser with the profile from prerequisites.
2. Navigate to "about:support" page.
3. Verify that the v.1.1.0rc1 add-on is enabled in the Extensions section.
4. Restart the browser.
5. Observe that the 1.1.0rc1 add-on is still enabled.

[Expected result]:

• The v.1.1.0rc1 add-on is still displayed as enabled in the Extensions section.

[Actual result]:

• The v.1.1.0rc1 add-on is no longer displayed in the Extensions section.

[Additional Notes]:

• We believe that the Balrog add-on somehow overwrites the v.1.1.0rc1 add-on from our Normandy testing recipe since the v.1.0.4 add-on is displayed as enabled in the Features section.
• The network.trr.mode pref remains set as 2 after 1.0.4 overwrites v.1.1.0rc1.
• There are no telemetry heuristics generated after 1.0.4 overwrites the v.1.1.0rc1.
• The Normandy recipe is still displayed as "Active" in the "about:studies" page.
• I have attached a screen recording of the issue:

[nhi-nguyen]

cc @mythmon

[mythmon]

I reproduced the issue following the STR in a new profile. I zipped up the testing profile.

doh-182-test.zip

[nhi-nguyen]

@SoftVision-CarmenFat: This is a blocking issue with the current deployment. We're planning to fix this issue by completely removing the addon from Balrog, then deploying v.1.0.4 to the users currently enrolled in the study - basically replacing Balrog with Normandy for those users. We would like to verify that this issue is not reproducible with the new deployment method. @wthayer will send out testing details via email.