Manual test run on ARM for 1.58.x - Nightly (Checking C116)

[LaurenWags]

Installer

• [x] Check that installer is close to the size of the last release
• [x] Check the Brave version in About and make sure it is EXACTLY as expected

Startup

• [x] Verify that Brave is only contacting *.brave.com endpoints on first launch using either Charles Proxy, Fiddler or Wireshark (or a similar application)
  • [x] Verify that opening a NTP doesn't trigger any outbound connections related to widgets without user interaction

Visual look

• [x] Make sure thereafter every merge
  • [x] No Chrome/Chromium words appear on brave://version
  • [x] No Chrome/Chromium words appear on normal or private tabs
  • [x] No Chrome/Chromium words appear in site settings for Location / Camera / Microphone / Augmented Reality
  • [x] No Chrome/Chromium icons are shown in normal or private tabs

Custom tabs

• [x] Make sure Brave handles links from Gmail, Slack
• [x] Ensure custom tabs work even with sync enabled/disabled

Tab Groups

• [x] Ensure tab-group is enabled by default
• [x] Verify toggling tab-group setting triggers a relaunch request
• [x] Verify tab-group is not reverted back to default setting after browser restart

Developer Tools

• [x] Verify you can inspect sub-links via dev tools

Clear Data

• [x] Verify Clear Data on exit works as intended

Settings and Bottom bar

• [x] Verify changing default settings are retained and don't cause the browser to crash
• [x] Verify bottom bar buttons (Home/Bookmark/Search/Tabs) work as expected
• [x] Verify items in hamburger menu/share menu to ensure nothing unexpected has been added

IPFS, ENS & Unstoppable Domain

• [x] Verify settings for IPFS, ENS & Unstoppable Domain show up under Brave Shields & Privacy settings

Downloads

• [x] Verify downloading a file works and that all actions on the download item work.
• [x] Verify that PDF is downloaded over HTTPS at https://basicattentiontoken.org/BasicAttentionTokenWhitePaper-4.pdf
• [x] Verify that PDF is downloaded over HTTP at http://www.pdf995.com/samples/pdf.pdf

Bravery settings

• [x] Check that HTTPS Everywhere works by loading http://https-everywhere.badssl.com/
• [x] Turning HTTPS Everywhere off and shields off both disable the redirect to https://https-everywhere.badssl.com/
• [x] Check that toggling to blocking and allow ads works as expected
• [x] Verify that clicking through a cert error in https://badssl.com/ works
• [x] Visit https://twitter.com/ and then turn on script blocking, nothing should load. Allow it from the script blocking UI in the URL bar and it should work.
• [x] Verify that default Bravery settings take effect on pages with no site settings
• [x] Verify that 3rd party storage results are blank at https://jsfiddle.net/7ke9r14a/7/ when 3rd party cookies are blocked
• [x] Choose a DNS provider from the list in Settings | Privacy | Use Secure DNS, load https://browserleaks.com/dns, and verify your ISP's DNS resolvers aren't detected and shown; only your chosen DoH provider should appear.
• [x] Run all six test configurations on https://dev-pages.brave.software/storage/ephemeral-storage.html and confirm results are as expected per each test listed.

  Fingerprint Tests

  • [x] Visit https://browserleaks.com/webrtc, ensure 2 blocked items are listed in shields
  • [x] Test that https://diafygi.github.io/webrtc-ips/ doesn't leak IP address for each option under Settings -> Privacy and Security -> WebRTC IP handling policy

Content Tests

• [x] Go to https://brianbondy.com/ and click on the twitter icon on the top right. Verify that context menus work in the new twitter tab
• [x] Go to https://feedly.com and make sure that the password can be saved. Make sure the saved password is auto-populated when you visit the site again
• [x] Verify that https://mixed-script.badssl.com/ shows up as grey not red (no mixed content scripts are run)

Brave Rewards

• [x] Verify you are able to create a new Rewards profile and are in the unverified state by default
  • [x] Verify when you visit a creator in this state the panel shows a prompt to connect a custodian and no BAT information (earnings, balance, etc) is displayed
  • [x] Verify when you visit brave://rewards in this state there are no Auto Contribution, Tipping, or summary panels on this page
  • [x] Verify you can toggle ads off/on from this page and the panel responds accordingly
• [x] Verify you are able to connect a custodian
  • [x] Verify Rewards balance shows correct BAT and USD value on brave://rewards and panel after you connect
  • [x] Verify when you click on the BR panel while on a site, the panel displays site-specific information (site favicon, domain, attention %)
  • [x] Verify BR panel and tip banner shows message about an unverified creator
  • [x] Verify you are able to tip a creator who has the same custodian as you
  • [x] Verify you are able to perform an auto contribution
  • [x] Verify if you disable auto-contribute you are still able to tip creators
  • [x] Verify if auto-contribute is disabled AC does not occur
  • [x] Verify that there is no "Disconnect" option once connected to a custodian
• [x] Verify that you are able to reset rewards

Brave Ads

• [x] Verify ads is auto-enabled when rewards is enabled for the supported region
• [x] Verify ads are only shown when the app is being used
• [x] Verify ad notification are shown based on ads per hour setting
• [x] Verify ad notifications stack up in notification tray
• [x] Verify swipe left/right dismisses the ad notification when shown and is not stored in the notification tray
• [x] Verify clicking on an ad notification shows the landing page
• [x] Verify view,clicked and landed and dismiss states are logged based on the action

Sync

• [x] Verify you are able to join sync chain by scanning the QR code
• [x] Verify you are able to join sync chain using code words
• [x] Verify you are able to create a sync chain on the device and add other devices to the chain via QR code/Code words
• [x] Verify once sync chain is created, Categories option is shown in devices list
• [x] Verify only Bookmarks is enabled in Categories by default
• [x] Verify enabling Sync everything enables all other switches
• [x] Verify existing bookmarks before joining sync chain also gets sync'd to all devices on the sync chain
• [x] Verify Autofill data is sync'd to all devices when enabled
• [x] Verify History is sync'd to all devices
• [x] Verify Open tab form other devices shows up under history below the device name
• [x] Verify Password is sync'd to all devices
• [x] Verify Settings sync's site settings on all devices
• [x] Verify sync works on an upgrade profile and new bookmarks added post-upgrade sync's across devices on the chain
• [x] Verify adding a bookmark on custom tab gets sync'd across all devices in the chain
• [x] Verify you are able to create a standalone sync chain with one device
• [x] Verify Remove this device shows a confirmation alert before removing the sync chain on the device

Top sites view

• [x] Long-press on top sites to get to deletion mode, and delete a top site (note this will stop that site from showing up again on top sites, so you may not want to do this a site you want to keep there)

Session storage

• [x] Verify that tabs restore when closed, including active tab

Upgrade from previous version

Examples of pre-requisites before upgrading from previous version to the build being tested:

• visit several websites so Top Tiles under New Tab Page get updated/populated
• add several websites to the Android home screen via Add to Home screen (from the hamburger/setting menu)
• add several bookmarks include folders
• change/update several settings (Example: default search engine, adding a custom home page, changing site settings, changing several privacy settings etc..)
• create/enable a Brave Wallet (or restore if individual who's going through the upgrade has one available)
• enable Brave News and add/remove several sources
• change the default shield settings on varios websites (Example: enable script blocking/change FP/Ad blocking to strict, disable shields)
• Login into several websites and leave them opened
• Login into several websites and close the tabs so they're not opened when upgrading
• Login into a website and leave it as the active tab while upgrading

• enable/create a sync chain with several devices

  Upgrade Cases

• [x] Ensure that brave://version displays both the correct Brave version and expected Chromium version
• [x] Ensure that bookmarks and folders from the previous release have been retained/can be loaded without issues
• [x] Ensure that previously opened tabs are retained and can be lazy loaded with out issues
  • [x] Ensure that websites that have been logged into and were opened while upgrading are still logged in (ensure cookies are not cleared/lost)
• [x] Ensure that websites that have been logged into but weren't open when upgrading are still logged in when loading in new tab (ensure cookies are not cleared/lost)
• [x] Ensure that the correct website loads as the active tab (as per the pre-requisites, this tab should be logged into a website)
• [x] Ensure that the shields settings from the previous versions are retained on each website
• [x] Ensure that saved passwords are being displayed under Settings -> Passwords and can be autofilled without issues
• [x] Ensure that the sync chain is preserved and syncing still works across the devices that have been added
  • [x] Ensure that sharing a links/tabs using Send to your device works as expected
• [x] Ensure that Brave News is still enabled and the sources changes from the previous release are retained
• [x] Ensure that the various setting changes via Settings are retained (Example: enable script blocking/change FP/Ad blocking to strict, disable shields)
  • [x] Ensure that IPFS, ENS and Unstoppable Domain settings are retained from the previous version
• [x] Ensure that History is retained from the previous version
• [x] Ensure that the website shortcuts added via Add to Home screen are still visible on the Android home screen and tapping on icons loads the appropriate webpage
• [x] Ensure that the Tab Group setting doesn't reset/change after upgrading
• [x] Ensure that Clear data on exit is retained from the previous version
• [x] Rewards
  • [x] BAT balance is retained
  • [x] Auto-contribute list is retained
  • [x] Both Tips and Monthly Contributions are retained
  • [x] Summary panel transactions list is retained
  • [x] Changes to rewards settings are retained
• [x] Ads
  • [x] Both Estimated pending rewards & Ad notifications received this month are retained
  • [x] Changes to ads settings are retained
  • [x] Ensure that ads are not being enabled when upgrading to a new version if they were disabled
  • [x] Ensure that ads are not disabled when upgrading to a new version if they were enabled

[hffvld]

Verified on Pixel 7 using version(s):

Device/OS: Pixel 7 [panther_beta-user 14 UPB4.230623.007 release-keys]
Brave build: 1.58.52 
Chromium: 116.0.5845.51 (Official Build) canary (64-bit) 
Revision: d5ec9991d084625358dae86ed02de8dc760d1985

Issue #31216 still exists

Filed new issue #31939, #31956, #31962, #31964