Upgrade from C127 -> C128 on macOS x64 via 1.70.x - Nightly (Checking C128)

LaurenWags commented 1 month ago

As per process, QA runs through the following cases to ensure that the major chromium bump that's about to get merged into master doesn't regress the Nightly channel. Once the major chromium bump is merged into master, QA will run through a full manual pass on Windows & Android.

Startup & Components

[x] Verify that Brave is only contacting *.brave.com endpoints on first launch using either Charles Proxy, Fiddler, Wireshark or LittleSnitch (or a similar application)
- [x] Verify that opening a NTP doesn't trigger any outbound connections related to widgets without user interaction
[x] Remove the following component folders and ensure that they're being re-downloaded after restarting the browser:
- [x] afalakplffnnnlkncjhbmahjfjhmlkal: AutoplayWhitelist.dat, ExtensionWhitelist.dat, ReferrerWhitelist.json and Greaselion.json
- [x] CertificateRevocation
- [x] gccbbckogglekeggclmmekihdgdpdgoe: (Sponsored New Tab Images)
- [x] gkboaolpopklhgplhaaiboijnklogmbc: Brave Ad Block List Catalog
- [x] iodkpdagapdfkphljnddpjlldadblomo: Brave Ad Block Updater
- [x] mfddibmblmbccpadfndgakiopmmhebop: Brave Ad Block List Catalog
- [x] oofiananboodjbbmdelgdommihjbkfag: HTTPSE
- [x] Safe Browsing
[x] Restart the browser, load brave://components, wait for 8 mins and verify that no component shows any errors

Note: Always double check brave://components to make sure there's no errors/missing version numbers

Upgrade

[x] Make sure that data from the last version appears in the new version OK
[x] Ensure that brave://version lists the expected Brave & Chromium versions
[x] With data from the last version, verify that:
- [x] Bookmarks on the bookmark toolbar and bookmark folders can be opened
- [x] Cookies are preserved
- [x] Installed extensions are retained and work correctly
- [x] Opened tabs can be reloaded
- [x] Stored passwords are preserved
- [x] Sync chain created in previous version is retained
- [x] Social media-blocking buttons changes are retained
- [x] Custom filters under brave://settings/shields/filters are retained
- [x] Custom lists under brave://settings/shields/filters are retained
- [x] Rewards
- [x] BAT balance is retained
- [x] Auto-contribute list is retained
- [x] Both Tips and Monthly Contributions are retained
- [x] Panel transactions list is retained
- [x] Changes to rewards settings are retained
- [x] Ensure that Auto Contribute is not being enabled when upgrading to a new version if AC was disabled
- [x] Ads
- [x] Both Estimated pending rewards & Ad notifications received this month are retained
- [x] Changes to ads settings are retained
- [x] Ensure that ads are not being enabled when upgrading to a new version if they were disabled
- [x] Ensure that ads are not disabled when upgrading to a new version if they were enabled

stephendonner commented 1 month ago

Verified `PASSED` using

Brave | 1.70.49 Chromium: 127.0.6533.88 (Official Build) nightly (x86_64)
-- | --
Revision | 5cd6f1ea4ad6a0740fa5a679147c45f95b577f95
OS | macOS Version 11.7.10 (Build 20G1427)

and

Brave | 1.70.54 Chromium: 128.0.6613.18 (Official Build) nightly (x86_64)
-- | --
Revision | 30140ea3aefda98e32ddd4daa2309ca5eb273f64
OS | macOS Version 11.7.10 (Build 20G1427)

C127 -> C128 on-demand upgrade check - `PASSED`

1. Install `C127` build (`1.70.49` from https://github.com/brave/brave-browser/releases/tag/v1.70.49) 2. Launch via terminal as follows: `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates.brave.com/sparkle/Brave-Browser/nothanks/appcast.xml` (this is so you don't pick up any newer Nightly versions that have already been uploaded to sparkle) 3. Watch logs, confirm no update found (can also visit `brave://settings/help` but close this tab prior to closing the browser) 4. Close `Brave` 5. Relaunch with `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates-panel.brave.com/sparkle/Brave-Browser/nightly/appcast.xml` 6. Navigate to `brave://settings/help` and ensured that an update has been found/being applied ``` [2599:259:0807/145946.277598:INFO:sparkle_glue.mm(323)] brave update: did finish loading appcast [2599:259:0807/145946.279069:INFO:sparkle_glue.mm(329)] brave update: did find valid update with AppcastItem(Date: Wed, 07 Aug 2024 18:08:45 +0000, Version: 1.70.54.0) [2599:259:0807/145946.280601:INFO:sparkle_glue.mm(350)] brave update: willDownloadUpdate with AppcastItem(Date: Wed, 07 Aug 2024 18:08:45 +0000, Version: 1.70.54.0) ``` 7. Confirm you are prompted to "Relaunch" to get update (see note in first case about this button) Screen Shot 2024-08-07 at 3 18 37 PM

Once relaunched, ensured the correct version was installed as per the following: **`Example`** | **`Example`** ----------------|---------------- Screen Shot 2024-08-07 at 3 29 01 PM

|

C128 -> C128 on-demand upgrade check - `PASSED`

1. Install `C128` PR build (x64 `1.70.54` builds available via https://bravesoftware.slack.com/archives/CU5UXM3EC/p1723054326627009?thread_ts=1722962730.738489&cid=CU5UXM3EC) 2. Launch via terminal as follows: `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates.brave.com/sparkle/Brave-Browser/nothanks/appcast.xml` (this is so you don't pick up any newer Nightly versions that have already been uploaded to sparkle) 3. Watch logs, confirm no update found (can also visit `brave://settings/help` but close this tab prior to closing the browser) 4. Close `Brave` 5. Note, if you want to check deltas, at this point open something like Fiddler Everywhere before continuing. 6. Relaunch with `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates-panel.brave.com/sparkle/Brave-Browser/nightly/appcast.xml` 7. Navigate to `brave://settings/help` and ensured that an update has been found/being applied ``` [7952:259:0807/234928.544666:INFO:sparkle_glue.mm(323)] brave update: did finish loading appcast [7952:259:0807/234928.546286:INFO:sparkle_glue.mm(329)] brave update: did find valid update with AppcastItem(Date: Wed, 07 Aug 2024 22:41:08 +0000, Version: 1.70.56.0) [7952:259:0807/234928.547151:INFO:sparkle_glue.mm(350)] brave update: willDownloadUpdate with AppcastItem(Date: Wed, 07 Aug 2024 22:41:08 +0000, Version: 1.70.56.0) [7952:259:0807/235010.266300:INFO:sparkle_glue.mm(386)] brave update: will install update on quit with AppcastItem(Date: Wed, 07 Aug 2024 22:41:08 +0000, Version: 1.70.56.0) ``` 8. Confirm you are prompted to "Relaunch" to get update (note, if you click "Relaunch" button you will relaunch and could potentially pick up a newer Nightly that is on the previous Chromium version. Closing and then relaunching via the terminal will not update you again) Once relaunched, ensured the correct version was installed as per the following: **`Example`** | **`Example`** ----------------|---------------- Screen Shot 2024-08-07 at 8 45 27 PM

|

C127 -> C128 background upgrade check - `PASSED`

1. Install `C127` build via https://github.com/brave/brave-browser/releases/tag/v1.70.49 2. Launch via terminal as follows: `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates.brave.com/sparkle/Brave-Browser/nothanks/appcast.xml` (this is so you don't pick up any newer Nightly versions that have already been uploaded to sparkle) 3. Watch logs, confirm no update found (can also visit `brave://settings/help` but be sure to close this tab prior to closing the browser) 4. Close browser 5. Delete the following: - `com.brave.Browser.nightly` folder from `~/Library/Caches` - `com.brave.Browser.nightly.plist` file from `~/Library/Preferences` - Alternatively you can probably advance your computer clock ~4 hours, I am fairly certain deleting at least one of the above files removes the timer check for background updates. 6. Restart your computer if you deleted files (if you just advanced computer clock this step isn't necessary) 7. After computer restart, Relaunch with `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates-panel.brave.com/sparkle/Brave-Browser/nightly-arm64/appcast.xml` 9. DO NOT navigate to `brave://settings/help` 10. Watch the logs and see that an update is found (be sure you have all of the below before proceeding) ``` [3609:259:0807/183939.946523:INFO:sparkle_glue.mm(323)] brave update: did finish loading appcast [3609:259:0807/183939.948258:INFO:sparkle_glue.mm(329)] brave update: did find valid update with AppcastItem(Date: Wed, 07 Aug 2024 18:08:45 +0000, Version: 1.70.54.0) [3609:259:0807/183939.948924:INFO:sparkle_glue.mm(350)] brave update: willDownloadUpdate with AppcastItem(Date: Wed, 07 Aug 2024 18:08:45 +0000, Version: 1.70.54.0) [3609:259:0807/184026.868253:INFO:sparkle_glue.mm(386)] brave update: will install update on quit with AppcastItem(Date: Wed, 07 Aug 2024 18:08:45 +0000, Version: 1.70.54.0) ``` 11. Close and restart the browser via terminal with same command as step 7 12. Check version is updated under `brave://version` Screen Shot 2024-08-07 at 6 41 45 PM

C128 -> C128 background upgrade check - `PASSED`

1. Install `C128` PR build (x64 `1.70.54` builds available via https://bravesoftware.slack.com/archives/CU5UXM3EC/p1723054326627009?thread_ts=1722962730.738489&cid=CU5UXM3EC) 2. Launch via terminal as follows: `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates.brave.com/sparkle/Brave-Browser/nothanks/appcast.xml` (this is so you don't pick up any newer Nightly versions that have already been uploaded to sparkle) 3. Watch logs, confirm no update found (can also visit `brave://settings/help` but close this tab prior to closing the browser) 4. Close `Brave` 5. Note, if you want to check deltas, at this point open something like Fiddler Everywhere before continuing. 6. Relaunch with `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates-panel.brave.com/sparkle/Brave-Browser/nightly/appcast.xml` 7. Navigate to `brave://settings/help` and ensured that an update has been found/being applied ``` [5600:259:0807/235343.875061:INFO:sparkle_glue.mm(323)] brave update: did finish loading appcast [5600:259:0807/235343.877459:INFO:sparkle_glue.mm(329)] brave update: did find valid update with AppcastItem(Date: Wed, 07 Aug 2024 22:41:08 +0000, Version: 1.70.56.0) [5600:259:0807/235343.878763:INFO:sparkle_glue.mm(350)] brave update: willDownloadUpdate with AppcastItem(Date: Wed, 07 Aug 2024 22:41:08 +0000, Version: 1.70.56.0) [5600:259:0807/235355.653901:INFO:sparkle_glue.mm(386)] brave update: will install update on quit with AppcastItem(Date: Wed, 07 Aug 2024 22:41:08 +0000, Version: 1.70.56.0) ``` 8. Confirm you are prompted to "Relaunch" to get update (note, if you click "Relaunch" button you will relaunch and could potentially pick up a newer Nightly that is on the previous Chromium version. Closing and then relaunching via the terminal will not update you again) Once relaunched, ensured the correct version was installed as per the following: **`Example`** | **`Example`** ----------------|---------------- Screen Shot 2024-08-07 at 11 56 18 PM

| |

LaurenWags commented 1 month ago

Upgrade checks (using arm64 .dmg):

Brave   1.70.50 Chromium: 128.0.6613.18 (Official Build) nightly (arm64) 
Revision    78c1a2769e6dcf20b8daf7a2c6b9b902db7ac4b3
OS  macOS Version 14.6 (Build 23G80)

and

Brave   1.70.53 Chromium: 128.0.6613.18 (Official Build) nightly (arm64) 
Revision    83a4be99f404ccc245c4f5edc4690baea5beb31c
OS  macOS Version 14.6 (Build 23G80)

Note, need to be on internal VPN to do the below upgrade checks.

C128 --> C128 on-demand upgrade check - PASSED

1. Install `C128` PR build (arm64 builds available via https://bravesoftware.slack.com/archives/CU5UXM3EC/p1722962810590859?thread_ts=1722962730.738489&cid=CU5UXM3EC) 2. Launch via terminal as follows: `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates.brave.com/sparkle/Brave-Browser/nothanks/appcast.xml` (this is so you don't pick up any newer Nightly versions that have already been uploaded to sparkle) 3. Watch logs, confirm no update found (can also visit `brave://settings/help` but close this tab prior to closing the browser) 4. Close `Brave` 5. Note, if you want to check deltas, at this point open something like Fiddler Everywhere before continuing. 6. Relaunch with `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates-panel.brave.com/sparkle/Brave-Browser/nightly-arm64/appcast.xml` 7. Navigate to `brave://settings/help` and ensured that an update has been found/being applied ``` [17477:259:0807/143442.139754:INFO:sparkle_glue.mm(323)] brave update: did finish loading appcast [17477:259:0807/143442.141286:INFO:sparkle_glue.mm(329)] brave update: did find valid update with AppcastItem(Date: Tue, 06 Aug 2024 20:46:05 +0000, Version: 1.70.53.0) [17477:259:0807/143442.142611:INFO:sparkle_glue.mm(350)] brave update: willDownloadUpdate with AppcastItem(Date: Tue, 06 Aug 2024 20:46:05 +0000, Version: 1.70.53.0) [17477:259:0807/143449.607017:INFO:sparkle_glue.mm(386)] brave update: will install update on quit with AppcastItem(Date: Tue, 06 Aug 2024 20:46:05 +0000, Version: 1.70.53.0) ``` 8. Confirm you are prompted to "Relaunch" to get update (note, if you click "Relaunch" button you will relaunch and could potentially pick up a newer Nightly that is on the previous Chromium version. Closing and then relaunching via the terminal will not update you again) ![Screenshot 2024-08-07 at 2 35 21 PM](https://github.com/user-attachments/assets/820a96c3-de82-4340-83aa-6c660604b6a6) Once relaunched, ensured the correct version was installed as per the following: **`Example`** | **`Example`** ----------------|---------------- ![Screenshot 2024-08-07 at 2 36 02 PM](https://github.com/user-attachments/assets/77e61794-c84f-471a-93d9-f62cde04c48b) | ![Screenshot 2024-08-07 at 2 36 09 PM](https://github.com/user-attachments/assets/86115a66-79fa-41f9-9e72-6e63c4d7a3f6) Confirmed deltas: **`Example`** | **`Example`** ----------------|---------------- ![Screenshot 2024-08-07 at 2 34 55 PM](https://github.com/user-attachments/assets/f89a6c28-37bf-4eac-9a28-028824a91fb7) | ![Screenshot 2024-08-07 at 2 35 08 PM](https://github.com/user-attachments/assets/72e679eb-3da9-4003-9728-eaf108fe04f7)

C127 --> C128 on-demand upgrade check - PASSED

1. Install `C127` build via https://github.com/brave/brave-browser/releases/tag/v1.70.49 2. Launch via terminal as follows: `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates.brave.com/sparkle/Brave-Browser/nothanks/appcast.xml` (this is so you don't pick up any newer Nightly versions that have already been uploaded to sparkle) 3. Watch logs, confirm no update found (can also visit `brave://settings/help` but close this tab prior to closing the browser) 4. Close `Brave` 5. Relaunch with `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates-panel.brave.com/sparkle/Brave-Browser/nightly-arm64/appcast.xml` 6. Navigate to `brave://settings/help` and ensured that an update has been found/being applied ``` [13087:259:0806/162332.804183:INFO:sparkle_glue.mm(323)] brave update: did finish loading appcast [13087:259:0806/162332.805559:INFO:sparkle_glue.mm(329)] brave update: did find valid update with AppcastItem(Date: Tue, 06 Aug 2024 16:49:21 +0000, Version: 1.70.50.0) [13087:259:0806/162332.807167:INFO:sparkle_glue.mm(350)] brave update: willDownloadUpdate with AppcastItem(Date: Tue, 06 Aug 2024 16:49:21 +0000, Version: 1.70.50.0) [13087:259:0806/162602.054678:INFO:sparkle_glue.mm(386)] brave update: will install update on quit with AppcastItem(Date: Tue, 06 Aug 2024 16:49:21 +0000, Version: 1.70.50.0) ``` 7. Confirm you are prompted to "Relaunch" to get update (see note in first case about this button) ![Screenshot 2024-08-06 at 4 26 12 PM](https://github.com/user-attachments/assets/13c5256d-ad9a-45c8-bd80-4f42f115e0a3) Once relaunched, ensured the correct version was installed as per the following: **`Example`** | **`Example`** ----------------|---------------- ![Screenshot 2024-08-06 at 4 26 54 PM](https://github.com/user-attachments/assets/a93ca5c1-a826-4cca-818c-15a7dfb83d8c) | ![Screenshot 2024-08-06 at 4 27 01 PM](https://github.com/user-attachments/assets/3d371fd8-38aa-45c6-8571-c4badd5fd4ee)

C128 --> C128 background upgrade check - PASSED

1. Install `C128` PR build (arm64 builds available via https://bravesoftware.slack.com/archives/CU5UXM3EC/p1722962810590859?thread_ts=1722962730.738489&cid=CU5UXM3EC) 2. Launch via terminal as follows: `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates.brave.com/sparkle/Brave-Browser/nothanks/appcast.xml` (this is so you don't pick up any newer Nightly versions that have already been uploaded to sparkle) 3. Watch logs, confirm no update found (can also visit `brave://settings/help` but be sure to close this tab prior to closing the browser) 4. Close browser 5. Delete the following: - `com.brave.Browser.nightly` folder from `~/Library/Caches` - `com.brave.Browser.nightly.plist` file from `~/Library/Preferences` - Alternatively you can probably advance your computer clock ~4 hours, I am fairly certain deleting at least one of the above files removes the timer check for background updates. 6. Restart your computer if you deleted files (if you just advanced computer clock this step isn't necessary) 7. Note, if you want to check deltas, at this point open something like Fiddler Everywhere before continuing. 8. After computer restart, Relaunch with `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates-panel.brave.com/sparkle/Brave-Browser/nightly-arm64/appcast.xml` 9. DO NOT navigate to `brave://settings/help` 10. Watch the logs and see that an update is found (be sure you have all of the below before proceeding) ``` [18225:259:0807/205319.850149:INFO:sparkle_glue.mm(323)] brave update: did finish loading appcast [18225:259:0807/205319.851358:INFO:sparkle_glue.mm(329)] brave update: did find valid update with AppcastItem(Date: Tue, 06 Aug 2024 20:46:05 +0000, Version: 1.70.53.0) [18225:259:0807/205319.853323:INFO:sparkle_glue.mm(350)] brave update: willDownloadUpdate with AppcastItem(Date: Tue, 06 Aug 2024 20:46:05 +0000, Version: 1.70.53.0) [18225:259:0807/205326.719149:INFO:sparkle_glue.mm(386)] brave update: will install update on quit with AppcastItem(Date: Tue, 06 Aug 2024 20:46:05 +0000, Version: 1.70.53.0) ``` 11. Close and restart the browser via terminal with same command as step 8 12. Check version is updated under `brave://version` ![Screenshot 2024-08-07 at 8 54 16 PM](https://github.com/user-attachments/assets/171e7e91-35c5-4fdd-802b-545c4edc7c30) 13. Confirmed deltas: Example | Example --- | --- ![Screenshot 2024-08-07 at 8 53 30 PM](https://github.com/user-attachments/assets/f58b0a70-2a1c-410c-8940-ac3b7c01994d) | ![Screenshot 2024-08-07 at 8 53 42 PM](https://github.com/user-attachments/assets/a981dadc-77ac-43d8-a521-aa1a0a00e8d2)

C127 --> C128 background upgrade check - PASSED

1. Install `C127` build via https://github.com/brave/brave-browser/releases/tag/v1.70.49 2. Launch via terminal as follows: `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates.brave.com/sparkle/Brave-Browser/nothanks/appcast.xml` (this is so you don't pick up any newer Nightly versions that have already been uploaded to sparkle) 3. Watch logs, confirm no update found (can also visit `brave://settings/help` but be sure to close this tab prior to closing the browser) 4. Close browser 5. Delete the following: - `com.brave.Browser.nightly` folder from `~/Library/Caches` - `com.brave.Browser.nightly.plist` file from `~/Library/Preferences` - Alternatively you can probably advance your computer clock ~4 hours, I am fairly certain deleting at least one of the above files removes the timer check for background updates. 6. Restart your computer if you deleted files (if you just advanced computer clock this step isn't necessary) 7. After computer restart, Relaunch with `/Applications/Brave\ Browser\ Nightly.app/Contents/MacOS/Brave\ Browser\ Nightly --args --enable-logging=stderr --update-feed-url=https://updates-panel.brave.com/sparkle/Brave-Browser/nightly-arm64/appcast.xml` 9. DO NOT navigate to `brave://settings/help` 10. Watch the logs and see that an update is found (be sure you have all of the below before proceeding) ``` [13388:259:0806/193240.393170:INFO:sparkle_glue.mm(323)] brave update: did finish loading appcast [13388:259:0806/193240.395045:INFO:sparkle_glue.mm(329)] brave update: did find valid update with AppcastItem(Date: Tue, 06 Aug 2024 16:49:21 +0000, Version: 1.70.50.0) [13388:259:0806/193240.396443:INFO:sparkle_glue.mm(350)] brave update: willDownloadUpdate with AppcastItem(Date: Tue, 06 Aug 2024 16:49:21 +0000, Version: 1.70.50.0) [13388:259:0806/193504.629285:INFO:sparkle_glue.mm(386)] brave update: will install update on quit with AppcastItem(Date: Tue, 06 Aug 2024 16:49:21 +0000, Version: 1.70.50.0) ``` 11. Close and restart the browser via terminal with same command as step 7 12. Check version is updated under `brave://version` ![Screenshot 2024-08-06 at 7 35 59 PM](https://github.com/user-attachments/assets/b6835bb0-991f-4804-b34d-db38409a7023)

stephendonner commented 1 month ago

Closed too soon; still need to do update checks here; reopening.

stephendonner commented 1 month ago

Done!

brave / brave-browser