brave / qa-resources

Mozilla Public License 2.0
21 stars 19 forks source link

Changing `pkg` -> `dmg` on `brave.com` #582

Closed kjozwiak closed 8 months ago

kjozwiak commented 8 months ago

Release Date/Target: February 28th, 2024 as previously discussed in several Slack DMs

Summary:

QA will run through the brave.com staging website via https://bravesoftware.com and ensure that the correct executables are being served for each platform. Specifically ensuring that pkg are being replaced with dmg for macOS. We'll also ensure that the referral codes are being added into brave://local-state and that there's no referral code within the executable name.

For each of the above checks, we'll ensure that:

Related GH issues/Documentation:

QA re: ensuring that the dmg executables were working was done via https://github.com/brave/brave-browser/issues/32316#issuecomment-1788407943, https://github.com/brave/brave-browser/issues/32316#issuecomment-1789601030 & https://github.com/brave/brave-browser/issues/32316#issuecomment-1789640524.

LaurenWags commented 8 months ago

Verified with macOS 14.x arm64 as per the below:

Safari - PASSED Using Safari (version 17.4 (19618.1.13.11.5)) - navigated to bravesoftware.com, entered credentials from 1PW - clicked on "Download" button on main page - confirmed downloaded the .dmg file, not .pkg - confirmed no ref code appended to the .dmg file - installed the .dmg file - confirmed able to install without issue - confirmed the .dmg installed the "Universal" version - confirmed current RC version 1.63.164 was installed - visited brave://local-state - confirmed expected ref code (BRV002) was listed - closed browser, deleted application, profile, and downloaded file - spot checked the above for other ways to download: - "Download" button in the page header - bravesoftware.com/download page Example | Example --- | --- ![Screenshot 2024-02-27 at 4 10 07 PM](https://github.com/brave/qa-resources/assets/28145373/fd885c6d-664c-4951-b8e4-2cddb5b08db7) | ![Screenshot 2024-02-27 at 4 11 04 PM](https://github.com/brave/qa-resources/assets/28145373/1e2aca00-62a8-462e-91d4-61e38f075659)
Chrome - PASSED Using Chrome (version 122.0.6261.94) - navigated to bravesoftware.com, entered credentials from 1PW - clicked on "Download" button on main page - confirmed downloaded the .dmg file, not .pkg - confirmed no ref code appended to the .dmg file - installed the .dmg file - confirmed able to install without issue - confirmed the .dmg installed the "Universal" version - confirmed current RC version 1.63.164 was installed - visited brave://local-state - confirmed expected ref code (BRV002) was listed - closed browser, deleted application, profile, and downloaded file - spot checked the above for other ways to download: - "Download" button in the page header - bravesoftware.com/download page Example | Example | Example --- | --- | --- ![Screenshot 2024-02-27 at 4 29 48 PM](https://github.com/brave/qa-resources/assets/28145373/ea01c70a-e10e-4088-8aa3-0824f1fd6265) | ![Screenshot 2024-02-27 at 4 30 14 PM](https://github.com/brave/qa-resources/assets/28145373/fe08ef1f-910e-4acf-b0a1-65550803f7ab) | ![Screenshot 2024-02-27 at 4 30 49 PM](https://github.com/brave/qa-resources/assets/28145373/a01a2aaf-7045-48ab-a74d-6abe0ef303df)
kjozwiak commented 8 months ago

macOS x64 running Sonoma 14.3.1 - PASSED

Chrome - PASSED * visited `https://bravesoftware.com` using `Chrome 122.0.6261.69 (Official Build) (x86_64)` * clicked on `Download Brave` that's in the middle of the page and ensured that a `universal dmg` was being downloaded * ensured that both `Download Brave` buttons were working correctly on `https://bravesoftware.com` * ensured that `Brave-Browser.dmg` was being downloaded **`Example`** | **`Example`** | **`Example`** ----------------|----------------|--------------- ![Screenshot 2024-02-27 at 4 20 49 PM](https://github.com/brave/qa-resources/assets/2602313/12c67b3b-f0c6-4b97-a1fd-1c7f3d877284) | ![Screenshot 2024-02-27 at 4 17 50 PM](https://github.com/brave/qa-resources/assets/2602313/f6838172-183a-4860-8cd0-9bfd8dda59de) | ![Screenshot 2024-02-27 at 4 16 38 PM](https://github.com/brave/qa-resources/assets/2602313/e92eee87-1aab-4129-a32a-b88ec2d1ef83) * once downloaded, ensured that installing `Brave` via `drag & drop` worked as expected and installed `Brave` * ensured that `1.63.164 Chromium: 122.0.6261.69` was being installed without any issues * ensured that the version that was installed was a `Universal` build * ensured that the `BRV002` referral code was being displayed via `brave://local-state` ``` "promo_code": { "metadata": [ "user_controlled", "user_modifiable", "extension_modifiable" ], "value": "BRV002" ``` **`Example`** | **`Example`** | **`Example`** ----------------|----------------|--------------- ![Screenshot 2024-02-27 at 4 18 22 PM](https://github.com/brave/qa-resources/assets/2602313/b3a7d5e4-34cf-4797-8d06-0c0f0326639f) | ![Screenshot 2024-02-27 at 4 19 04 PM](https://github.com/brave/qa-resources/assets/2602313/6d5e96ef-3ed7-4645-b33e-f21cefe2cbf4) | ![Screenshot 2024-02-27 at 4 26 12 PM](https://github.com/brave/qa-resources/assets/2602313/9c9d514e-006b-44d7-a16f-b6af20ba4cfc)
Firefox - PASSED * visited `https://bravesoftware.com` using `Firefox 123.0` * clicked on `Download Brave` that's in the middle of the page and ensured that a `universal dmg` was being downloaded * ensured that both `Download Brave` buttons were working correctly on `https://bravesoftware.com` * ensured that `Brave-Browser.dmg` was being downloaded **`Example`** | **`Example`** ----------------|--------------- ![Screenshot 2024-02-27 at 4 34 14 PM](https://github.com/brave/qa-resources/assets/2602313/3a375daa-243f-4121-af12-00dce7e59097) | ![Screenshot 2024-02-27 at 4 34 24 PM](https://github.com/brave/qa-resources/assets/2602313/56c98010-67e5-4fa8-a4c0-bb3c4a0e486f) * once downloaded, ensured that installing `Brave` via `drag & drop` worked as expected and installed `Brave` * ensured that `1.63.164 Chromium: 122.0.6261.69` was being installed without any issues * ensured that the version that was installed was a `Universal` build * ensured that the `BRV002` referral code was being displayed via `brave://local-state` ``` "promo_code": { "metadata": [ "user_controlled", "user_modifiable", "extension_modifiable" ], "value": "BRV002" ``` **`Example`** | **`Example`** | **`Example`** ----------------|----------------|--------------- ![Screenshot 2024-02-27 at 4 35 32 PM](https://github.com/brave/qa-resources/assets/2602313/2faca0d7-8566-4a70-92fd-556ac6214307) | ![Screenshot 2024-02-27 at 4 35 57 PM](https://github.com/brave/qa-resources/assets/2602313/9a40f277-10d7-4c70-8697-a794bc2e5bd1) | ![Screenshot 2024-02-27 at 4 36 12 PM](https://github.com/brave/qa-resources/assets/2602313/bb8723d4-41b2-4ee4-9b65-8662f2be673a)

Windows 11 x64 - PASSED

Chrome - PASSED * visited `https://bravesoftware.com` using `Chrome 122.0.6261.70 (Official Build) (64-bit) (cohort: Stable)` * clicked on `Download Brave` that's in the middle of the page * ensured that both `Download Brave` buttons were working correctly on `https://bravesoftware.com` * ensured that `BraveBrowserSetup-BRV002.exe` was being downloaded ![image](https://github.com/brave/qa-resources/assets/2602313/9e6befe4-e7ef-42a6-bd5f-5e45cb5d1ccd) * once downloaded, double clicked on `BraveBrowserSetup-BRV002.exe` and ensured that the `stub` installer was started * ensured that `1.63.162 Chromium: 122.0.6261.69` was installed without any issues * ensured that the version that was installed was a `Universal` build * ensured that the `BRV002` referral code was being displayed via `brave://local-state` ``` "promo_code": { "metadata": [ "user_controlled", "user_modifiable", "extension_modifiable" ], "value": "BRV002" ``` **`Example`** | **`Example`** | **`Example`** | **`Example`** ----------------|----------------|---------------|----------------- ![image](https://github.com/brave/qa-resources/assets/2602313/fd7c3f1f-1c72-45fd-a03e-7db7a19beb21) | ![image](https://github.com/brave/qa-resources/assets/2602313/f7661786-bf82-48e0-bb7e-7edb94e8e50b) | ![image](https://github.com/brave/qa-resources/assets/2602313/db1334a6-15a2-4da4-a91e-0035e8b93335) | ![image](https://github.com/brave/qa-resources/assets/2602313/a5aeea69-7400-40dd-883f-52956728b6e8)
Firefox- PASSED * visited `https://bravesoftware.com` using `Firefox 123.0` * clicked on `Download Brave` that's in the middle of the page * ensured that both `Download Brave` buttons were working correctly on `https://bravesoftware.com` * ensured that `BraveBrowserSetup-BRV002.exe` was being downloaded **`Example`** | **`Example`** ----------------|--------------- ![image](https://github.com/brave/qa-resources/assets/2602313/27679d88-36ee-4dde-b91b-e21e81deb97c) | ![image](https://github.com/brave/qa-resources/assets/2602313/0aa6b320-71df-490d-b343-3dbdd79171a1) * once downloaded, double clicked on `BraveBrowserSetup-BRV002.exe` and ensured that the `stub` installer was started * ensured that `1.63.162 Chromium: 122.0.6261.69` was installed without any issues * ensured that the version that was installed was a `Universal` build * ensured that the `BRV002` referral code was being displayed via `brave://local-state` ``` "promo_code": { "metadata": [ "user_controlled", "user_modifiable", "extension_modifiable" ], "value": "BRV002" ``` **`Example`** | **`Example`** | **`Example`** | **`Example`** ----------------|----------------|---------------|----------------- ![image](https://github.com/brave/qa-resources/assets/2602313/636bd5e5-a1be-40ae-8c78-5d041a9548c2) | ![image](https://github.com/brave/qa-resources/assets/2602313/40e20d12-f379-44e9-956d-667d69158f56) | ![image](https://github.com/brave/qa-resources/assets/2602313/476d6f3f-6675-499a-ba08-84a053aded6a) | ![image](https://github.com/brave/qa-resources/assets/2602313/0ec55ef3-ebf0-466b-8a2b-540db9b8e3b1)

Linux x64 running Ubuntu 22.04 x64 - PASSED

Chrome - PASSED * visited `https://bravesoftware.com` using `122.0.6261.94 (Official Build) (64-bit)` * clicked on `Download Brave` that's in the middle of the page * ensured that both `Download Brave` buttons were working correctly on `https://bravesoftware.com` * ensured that the user is redirected to `https://brave.com/linux` **`Example`** | **`Example`** ----------------|--------------- ![image](https://github.com/brave/qa-resources/assets/2602313/c9007695-0868-4800-bfa5-8315b45803de) | ![image](https://github.com/brave/qa-resources/assets/2602313/a75d0bd8-ddc4-453b-a055-9e225feae6ef)
Firefox - PASSED * visited `https://bravesoftware.com` using `Firefox 123.0` * clicked on `Download Brave` that's in the middle of the page * ensured that both `Download Brave` buttons were working correctly on `https://bravesoftware.com` * ensured that the user is redirected to `https://brave.com/linux` **`Example`** | **`Example`** ----------------|--------------- ![image](https://github.com/brave/qa-resources/assets/2602313/7643d699-d4d6-48db-8780-f2c04dbbe742) | ![image](https://github.com/brave/qa-resources/assets/2602313/af4aa635-cf17-4838-8bd4-e9c17f9159f8)
LaurenWags commented 8 months ago

Verified with Win 10 x86 as per the below:

Edge - PASSED Using Edge (version 122.0.2365.59) - navigated to bravesoftware.com, entered credentials from 1PW - clicked on "Download" button on main page - confirmed downloaded the expected .exe file - confirmed ref code appended to the .exe file - installed the .exe file - confirmed able to install without issue - confirmed the .exe file installed the x86 specific version for Windows - confirmed current RC version 1.63.164 was installed (note, got 1.63.162, confirming if this is OK) - visited brave://local-state - confirmed expected ref code (BRV002) was listed - Using a fresh VM, spot checked the above for other ways to download: - "Download" button in the page header Example | Example | Example --- | --- | --- Screenshot 2024-02-27 at 4 49 18 PM | Screenshot 2024-02-27 at 4 52 23 PM | Screenshot 2024-02-27 at 4 54 49 PM
Firefox - PASSED Using Firefox (version 121.0) - navigated to bravesoftware.com, entered credentials from 1PW - clicked on "Download" button on main page - confirmed downloaded the expected .exe file - confirmed ref code appended to the .exe file - installed the .exe file - confirmed able to install without issue - confirmed the .exe file installed the x86 specific version for Windows - confirmed current RC version 1.63.164 was installed (note, got 1.63.162, confirming if this is OK) - visited brave://local-state - confirmed expected ref code (BRV002) was listed - Using a fresh VM, spot checked the above for other ways to download: - bravesoftware.com/download page Example | Example | Example --- | --- | --- Screenshot 2024-02-27 at 5 03 32 PM | Screenshot 2024-02-27 at 5 04 43 PM | Screenshot 2024-02-27 at 5 05 10 PM
kjozwiak commented 8 months ago

Closing as verifications have been completed as per the above. Thanks for the help @LaurenWags 👍