Update this repo to use the new idempotent mazerunner API so that we can reduce the number of flakes.
Design
This follows the same approach as used in bugsnag-cocoa-performance, with the added twist that we store the most recent maze runner command UUID in NSUserDefaults so that the data survives app restarts (because we are deliberately crashing the app).
Changes
This PR unfortunately has a huge number of changed files, but most of them are very small, repetitive changes.
Every scenario has been modified to override configure instead of configuring in the startBugsnag method. This accounts for most of the changed files. Some scenarios have been updated to become more robust to the new timing.
The various .feature files needed to be updated for the different timing we now have, and to remove some old workarounds that are no longer relevant.
macOSTestApp and iosTestApp saw the most big changes in order to make them use the Fixture class rather than driving things itself.
eventMode has been removed, and instead every command takes a list of arguments. For example in AppHangScenario.swift, TimeInterval(eventMode!)! is now TimeInterval(args[0])!.
performBlockAndWaitForEventDelivery has been replaced with waitForEventDelivery:andThen because the old approach blocked the main thread, which can cause the app to hang. Similarly for performBlockAndWaitForSessionDelivery
All scenario and fixture configuration has been moved into FixtureConfig.
CommandReaderThread.swift, Fixture.swift, and MazeRunnerCommand.swift are copied from BugsnagPerformance with a few minor changes.
Testing
There is no new functionality - We're just changing the testing rig.
Goal
Update this repo to use the new idempotent mazerunner API so that we can reduce the number of flakes.
Design
This follows the same approach as used in bugsnag-cocoa-performance, with the added twist that we store the most recent maze runner command UUID in
NSUserDefaults
so that the data survives app restarts (because we are deliberately crashing the app).Changes
This PR unfortunately has a huge number of changed files, but most of them are very small, repetitive changes.
configure
instead of configuring in thestartBugsnag
method. This accounts for most of the changed files. Some scenarios have been updated to become more robust to the new timing..feature
files needed to be updated for the different timing we now have, and to remove some old workarounds that are no longer relevant.macOSTestApp
andiosTestApp
saw the most big changes in order to make them use the Fixture class rather than driving things itself.eventMode
has been removed, and instead every command takes a list of arguments. For example inAppHangScenario.swift
,TimeInterval(eventMode!)!
is nowTimeInterval(args[0])!
.performBlockAndWaitForEventDelivery
has been replaced withwaitForEventDelivery:andThen
because the old approach blocked the main thread, which can cause the app to hang. Similarly forperformBlockAndWaitForSessionDelivery
FixtureConfig
.CommandReaderThread.swift
,Fixture.swift
, andMazeRunnerCommand.swift
are copied from BugsnagPerformance with a few minor changes.Testing
There is no new functionality - We're just changing the testing rig.