Manual test run on Android ARM for 1.37.x - Release

LaurenWags commented 2 years ago

Installer

[x] Check that installer is close to the size of the last release
[x] Check the Brave version in About and make sure it is EXACTLY as expected

Startup

[x] Verify that Brave is only contacting *.brave.com endpoints on first launch using either Charles Proxy, Fiddler or Wireshark (or a similar application)
- [x] Verify that opening a NTP doesn't trigger any outbound connections related to widgets without user interaction

Custom tabs

[x] Make sure Brave handles links from Gmail, Slack
[x] Make sure Brave works as custom tabs provide with Chrome browser
[x] Ensure custom tabs work even with sync enabled/disabled

Settings and Bottom bar

[x] Verify changing default settings are retained and don't cause the browser to crash
[x] Verify bottom bar buttons (Home/Bookmark/Search/Tabs) work as expected

Rewards

[x] Verify account balance shows correct BAT and USD value
[x] Verify actions taken (claiming grant, tipping, auto-contribute) display in wallet panel
[x] Verify when you click on the BR panel while on a site, the panel displays site specific information (site favicon, domain, attention %)
[x] Verify you are able to make one-time tip and they display in tips panel
[x] Verify you are able to make recurring tip and they display in tips panel
[x] Verify you can tip a verified publisher
[x] Verify you can tip a verified YouTube creator
[x] Verify you are able to perform a contribution
[x] Verify if you disable auto-contribute you are still able to tip regular sites and YouTube creators
[x] Verify on BR panel that the Verify wallet button only enables when balance is >=15 BAT
[x] Verify you can reset rewards from advance setting. Resetting should delete wallet and bring it back to the pre-optin state

Brave Shields

[x] Run all six test configurations on https://dev-pages.brave.software/storage/ephemeral-storage.html and confirm results are as expected per each test listed.

Brave Ads

[x] Verify ads is auto-enabled when rewards is enabled for the supported region
[x] Verify ads are only shown when the app is being used
[x] Verify ad notification are shown based on ads per hour setting
[x] Verify ad notifications stack up in notification tray
[x] Verify swipe left/right dismisses the ad notification when shown and is not stored in the notification tray
[x] Verify clicking on an ad notification shows the landing page
[x] Verify view,clicked and landed and dismiss states are logged based on the action

Upgrade

Pre-Requisite: Visit several websites so Top Tiles under New Tab Page get updated and add several sites via Add to Home screen (from 3 dots menu) using a previous build/installation before upgrading.

[x] Make sure that data from the last version appears in the new version OK
[x] Ensure that brave://version lists the expected Brave & Chromium versions
[x] With data from the last version, verify that
- [x] Bookmarks are preserved (Folders/Sites)
- [x] Cookies are preserved
- [x] Per site shield settings are preserved
- [x] Opened tabs can be reloaded
- [x] Stored passwords are preserved
- [x] Sync chain created in previous version is retained
- [x] Setting changes are preserved (Default SE, Shield settings, Homepage etc..)
- [x] Top Tiles under New Tab Page are preserved
- [x] Home screen shortcuts are preserved
- [x] Rewards
- [x] Wallet balance is retained
- [ ] Auto-contribute list is retained
- [x] Both Tips and Monthly Contributions are retained
- [x] Wallet panel transactions list is retained
- [x] Changes to rewards settings are retained
- [x] Ads
- [x] Both Estimated pending rewards & Ad notifications received this month are retained
- [x] Changes to ads settings are retained
- [x] Ensure that ads are not being enabled when upgrading to a new version if they were disabled
- [x] Ensure that ads are not disabled when upgrading to a new version if they were enabled