Added a few more tests to mock test failure scenarios and fixed a few final Exit Status issues

[ravimandala]

When a test times out, fails, crashes the app or successfully passes the bundle exits with an appropriate exit status. Often times, one or more tests in the bundle can exhibit one or more of these behaviors in one single execution of a test bundle. However, the testing of this part of Bluepill's logic is limited right now partially because the combinations are exponential. Another reason is the lack of an easy-to-use mechanism to reproduce/mock different scenarios.

This PR is an attempt to add that ability along with a bunch of new tests and fix any bugs discovered.

With the new ability to test multiple cascading failure scenarios, a few more tests were added to uncover some known issues in exit status.

1. Bluepill tries to merge exit statuses with a | operator but since the exit status codes are not mergeable it is resulting in unexpected behavior. For example, if a bundle has BPExitStatusTestsFailed and BPExitStatusTestTimeout, then the final exit status is returned as BPExitStatusAppCrashed because 6 | 1 = 7. Fix: Change the Exit Status codes to powers of 2 to make them mergeable.

2. The exit status codes are not always merged in the same way making it difficult and complicated to predict the behavior. Solution: Simplified the Exit Status consolidation/aggregation logic. In case of failure, report all exit statuses that happened over multiple attempts.

3. The hasRemainingTestsInContext() is not capable of delivering the exact precise answer to the question... Are there more tests left to execute? partially because of a known issue (https://github.com/linkedin/bluepill/issues/352). So, depending on that is sometimes leading to an extra simulator attempt even after there are no tests left to execute. Fix: Considering the return of BPExitStatusTestsAllPassed an indication from Simulator that "all tests there are to execute are done". Another indication to end testing is when certain error occur and there are no more retries left.

\cc @ob @chenxiao0228

[ravimandala]

@ob @chenxiao0228 Let me fix the failing test. Please hold on reviewing this.

[ravimandala]

@ob @chenxiao0228 This is ready for review. Can you please take a look?

[ravimandala]

@oliverhu Can you please review this and let me know your comments? Thanks. \cc @chenxiao0228 @ob

[oliverhu]

hey @ravimandala , actually the test time jumped from 20 minutes to ONE HOUR?

https://github.com/linkedin/bluepill/runs/528445755 vs https://github.com/linkedin/bluepill/runs/576277516

[ravimandala]

@oliverhu The new tests added ~27 minutes. Most of it is simulator recreation for retries. I will try to reduce the test_timeouts/stuck_timeouts if possible.

New enabled/added BluepillTests
    ✓ testAppThatCrashesOnLaunch (281.371 seconds)
    ✓ testAppThatHangsOnLaunch (241.364 seconds)
    ✓ testReportFailureOnCrashTimeoutAndPass (165.264 seconds)
    ✓ testReportFailureOnFailTimeoutAndPass (40.480 seconds)
    ✓ testReportFailureOnTimeoutAndNoRetry (134.613 seconds)
    ✓ testReportFailureOnTimeoutCrashAndPass (166.021 seconds)
    ✓ testReportSuccessOnFailedTestAndPassOnRetryAll (100.243 seconds)
    ✓ testReportSuccessOnFailTimeoutAndPass (178.100 seconds)
    ✓ testReportSuccessOnTestFailedAndPassOnRetry (147.113 seconds)
    ✓ testReportSuccessOnTimeoutAndPassOnRetry (116.319 seconds)
    ✓ testRunningAndIgnoringCertainTestCases (42.885 seconds)
    ✓ testRunningOnlyCertainTestcases (55.221 seconds)

[ravimandala]

@ob Updated the README and option prompt. Added a section for exit codes and a note about the app crash. I hope it's looking good.

@oliverhu I reduced some time on the tests but not back to where it was because we have some very important tests and will be getting more. I hope it's fine.

[ravimandala]

Thanks, @oliverhu. Merging it now.