search

theappbusiness / TABTestKit

Library designed to make writing and maintaining automated tests for iOS applications. This includes automation of bio-metrics and controlling of mock servers
MIT License
59 stars 13 forks source link

Test for the slider fails on the example app #23

Closed TABanniefung closed 4 years ago

TABanniefung commented 4 years ago

Issue on the slider:

This step here Then(I: adjust(otherElementsScreen.slider, to: 1)) provided in the example fails.

The reason for the assertion failure is Element did not have the right value before timing out! Expected: 1.0, actual: 0.9971751570701599

KaneCheshire commented 4 years ago

I haven't been able to reproduce this @TABanniefung, what simulator/iOS version are you running this on for this to happen?

TABanniefung commented 4 years ago

it is failing on iphone XR OS 12.4

This might be an XCTest bug/limitation instead, according to https://metova.com/a-guide-to-xcode-ui-test/ , Asserting Adjusted Slider Values adjustToNormalizedSliderPosition does not guarantee accuracy but rather is an estimate. I found setting 0.0 or 1.0 does give 0% and 100% respectively but otherwise it is best to XCTAssertNotEqual to the previous value rather than test the new value.

zacoid55 commented 4 years ago

@TABanniefung could you give this another go, please? You'll need to be pointing to develop (and pull latest changes) like you were doing before we released recently

KaneCheshire commented 4 years ago

Fixing this in #38, by just removing the assertions. Apple say they can't guarantee it will be exact and I've started running into this issue too.

I tried setting the simulator to different resolutions in case it was an issue being on a retina Mac screen but didn't seem to make any difference, and it wouldn't really be fair to expect people to have to set that just to get tests to pass.