search

mozilla-mobile / focus-ios

⚠️ Firefox Focus (iOS) has moved to a new repository. It is now developed and maintained as part of: https://github.com/mozilla-mobile/firefox-ios
Mozilla Public License 2.0
1.26k stars 263 forks source link

Intermittent test failures - BasicBrowsing.testAdBlocking() #1894

Open isabelrios opened 3 years ago

isabelrios commented 3 years ago

Bitrise Test Run:

Provide a Bitrise test run report link here showcasing the problem https://app.bitrise.io/build/61f9ebef8d5af5fd#?tab=log

Stacktrace:

file:///Users/vagrant/git/XCUITest/BasicBrowsing.swift:CharacterRangeLen=0&EndingLineNumber=40&StartingLineNumber=40 - XCTAssertTrue failed

Build:

Main Jun 4th 9f73552

Looks like the site did not detect the Ads Blocking enabled. This is the screenshot on BR: Screenshot_27B9C440-359C-44F2-AB9D-17CFF422DB7C

And this is how it should look like: Simulator Screen Shot - iPhone 8 - 2021-06-04 at 10 34 41

┆Issue is synchronized with this Jira Task

isabelrios commented 3 years ago

@st3fan, @SimonBasca do you think there might be a real issue here? not sure why it detects to block ads sometimes and some other it fails to detect it... does not look like a timing issue, does it? I see this test as the one that fails more frequently

SimonBasca commented 3 years ago

We should change the test page. So far testing this manually I was unable to reproduce the issue.