[SR-14655] [Source Compat] RxTest - several errors related to XCTest usage

[benlangmuir]

Previous ID	SR-14655
Radar	rdar://78419005
Original Reporter	@benlangmuir
Type	Bug

Additional Detail from JIRA

| | | |------------------|-----------------| |Votes | 0 | |Component/s | | |Labels | Bug | |Assignee | None | |Priority | Medium | md5: 5feab7218a5310806d4d21013d2df92d

Issue Description:

Jenkins URL: https://ci.swift.org/job/swift-main-source-compat-suite/5899

/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:28:22: error: cannot convert value of type '[AnyEquatable<Event<Element>>]' to expected argument type '[CompletableEvent]'
      XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                     ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:28:22: note: arguments to generic parameter 'Element' ('AnyEquatable<Event<Element>>' and 'CompletableEvent') are expected to be equal
      XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                     ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:28:37: error: cannot convert value of type '[AnyEquatable<Event<Element>>]' to expected argument type '[CompletableEvent]'
      XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                                    ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:28:37: note: arguments to generic parameter 'Element' ('AnyEquatable<Event<Element>>' and 'CompletableEvent') are expected to be equal
      XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                                    ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:54:24: error: cannot convert value of type '[AnyEquatable<SingleEvent<Element>>]' to expected argument type '[CompletableEvent]'
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                       ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:54:24: note: arguments to generic parameter 'Element' ('AnyEquatable<SingleEvent<Element>>' and 'CompletableEvent') are expected to be equal
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                       ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:54:39: error: cannot convert value of type '[AnyEquatable<SingleEvent<Element>>]' to expected argument type '[CompletableEvent]'
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                                      ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:54:39: note: arguments to generic parameter 'Element' ('AnyEquatable<SingleEvent<Element>>' and 'CompletableEvent') are expected to be equal
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                                      ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:80:24: error: cannot convert value of type '[AnyEquatable<MaybeEvent<Element>>]' to expected argument type '[CompletableEvent]'
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                       ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:80:24: note: arguments to generic parameter 'Element' ('AnyEquatable<MaybeEvent<Element>>' and 'CompletableEvent') are expected to be equal
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                       ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:80:39: error: cannot convert value of type '[AnyEquatable<MaybeEvent<Element>>]' to expected argument type '[CompletableEvent]'
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                                      ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:80:39: note: arguments to generic parameter 'Element' ('AnyEquatable<MaybeEvent<Element>>' and 'CompletableEvent') are expected to be equal
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                                      ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:106:24: error: cannot convert value of type '[AnyEquatable<CompletableEvent>]' to expected argument type '[CompletableEvent]'
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                       ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:106:24: note: arguments to generic parameter 'Element' ('AnyEquatable<CompletableEvent>' and 'CompletableEvent') are expected to be equal
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                       ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:106:39: error: cannot convert value of type '[AnyEquatable<CompletableEvent>]' to expected argument type '[CompletableEvent]'
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                                      ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:106:39: note: arguments to generic parameter 'Element' ('AnyEquatable<CompletableEvent>' and 'CompletableEvent') are expected to be equal
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                                      ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:134:22: error: cannot convert value of type '[AnyEquatable<Recorded<Event<Element>>>]' to expected argument type '[CompletableEvent]'
      XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                     ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:134:22: note: arguments to generic parameter 'Element' ('AnyEquatable<Recorded<Event<Element>>>' and 'CompletableEvent') are expected to be equal
      XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                     ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:134:37: error: cannot convert value of type '[AnyEquatable<Recorded<Event<Element>>>]' to expected argument type '[CompletableEvent]'
      XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                                    ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:134:37: note: arguments to generic parameter 'Element' ('AnyEquatable<Recorded<Event<Element>>>' and 'CompletableEvent') are expected to be equal
      XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                                    ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:163:24: error: cannot convert value of type '[AnyEquatable<Recorded<Event<Element?>>>]' to expected argument type '[CompletableEvent]'
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                       ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:163:24: note: arguments to generic parameter 'Element' ('AnyEquatable<Recorded<Event<Element?>>>' and 'CompletableEvent') are expected to be equal
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                       ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:163:39: error: cannot convert value of type '[AnyEquatable<Recorded<Event<Element?>>>]' to expected argument type '[CompletableEvent]'
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                                      ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:163:39: note: arguments to generic parameter 'Element' ('AnyEquatable<Recorded<Event<Element?>>>' and 'CompletableEvent') are expected to be equal
        XCTAssertEqual(leftEquatable, rightEquatable, file: file, line: line)
                                      ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:188:9: error: cannot find 'XCTFail' in scope
        XCTFail("A non-next stop event has been emitted: \(stopEvent)", file: file, line: line)
        ^~~~~~~
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:204:20: error: cannot convert value of type '[Element]' to expected argument type '[CompletableEvent]'
    XCTAssertEqual(streamElements, elements, file: file, line: line)
                   ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:204:20: note: arguments to generic parameter 'Element' ('Element' and 'CompletableEvent') are expected to be equal
    XCTAssertEqual(streamElements, elements, file: file, line: line)
                   ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:204:36: error: cannot convert value of type '[Element]' to expected argument type '[CompletableEvent]'
    XCTAssertEqual(streamElements, elements, file: file, line: line)
                                   ^
/Users/buildnode/jenkins/workspace/swift-main-source-compat-suite/swift-source-compat-suite/project_cache/RxSwift/RxTest/XCTest+Rx.swift:204:36: note: arguments to generic parameter 'Element' ('Element' and 'CompletableEvent') are expected to be equal
    XCTAssertEqual(streamElements, elements, file: file, line: line) 

[benlangmuir]

XFAIL: https://github.com/apple/swift-source-compat-suite/pull/542

[hamishknight]

Looks like more configurations of this project are failing with same error in:
https://ci.swift.org/job/swift-main-source-compat-suite/6084/
https://ci.swift.org/job/swift-main-source-compat-suite-debug/4198/
https://ci.swift.org/job/swift-5.5-source-compat-suite/139/

XFAILing in https://github.com/apple/swift-source-compat-suite/pull/576

[benlangmuir]

Still XFAIL'd on main @ 8edaf2accf0de5

[justice-adams-apple]

This is happening because of the xcode 12.5 update which deprecates XCTest’s legacy Swift overlay library.

See https://developer.apple.com/documentation/xcode-release-notes/xcode-12_5-release-notes

I managed to confirm that building the project at the specified revision with ENABLE_TESTING_SEARCH_PATHS and removing -weak-lswiftXCTest allows the project to pass. I noticed this was actually fixed in RxSwift with https://github.com/ReactiveX/RxSwift/commit/b56f7a426e0c4bf4a5a2caec5dfd80fd347f9686#

This will require a project revision update for RxSwift within the source compatibility suite

[justice-adams-apple]

Hi @kzaher 👋 , as mentioned above, the RxSwift project is currently failing to build within the source compatibility suite due to an outdated revision.

I'd recommend updating the project definition in the source compat suite if you want your project to be continuously tested with compiler changes properly.

https://github.com/apple/swift-source-compat-suite/blob/main/projects.json#L2405

[freak4pc]

Hey @justice-adams-apple - I can help. Can you tell me exactly what changes need to be made?

Thanks,.

[justice-adams-apple]

:wave: @freak4pc I'd recommend opening a PR against https://github.com/apple/swift-source-compat-suite and update the following line which specified the revision of RxSwift to build within the source compatibility suite

https://github.com/apple/swift-source-compat-suite/blob/main/projects.json#L2405

Currently, we are using an outdated revision 70b8a3

What revision you update to is up to you (most people just take the head of main at the time of the PR). This will require you to update some of the associated xfails, and if you feel it necessary, you may want to update the project maintainer listed here: https://github.com/apple/swift-source-compat-suite/blob/main/projects.json#L2402

This will ensure your project is tested correctly as we continue making changes to swift. If you have any questions about changes, feel free to open a PR at swift-source-compat-suite and tag me directly and I'll get back to you

[freak4pc]

@justice-adams-apple thanks! is there any way to test locally that my changes are correct, or I'd have to open a PR to test them?

[justice-adams-apple]

Yeah you can test it locally, https://github.com/apple/swift-source-compat-suite#building-projects should give you some more insight into building your project locally. You'll likely want to run something along the lines of

./runner.py  --projects projects.json --include-actions 'action.startswith("Build")' --include-repos 'path == "RxSwift"'  --swiftc <YOUR_SWIFTC_PATH> 

[justice-adams-apple]

@freak4pc 👋 Hi , just wanted to ping you again. We typically ask that authors update their project within two weeks of being notified of an outdated source issue. Generally after this time period we simply remove said outdated project from the suite. Let me know if you still plan to update the RxSwift revision.

If you want to remove the RxSwift project form the source compatibility suite feel free to let me know and I can remove it. If you don't have the time currently, you are more than welcome to re-add the project to the compatibility suite at a later time if you decide to do so.

-Justice

[freak4pc]

Hey @justice-adams-apple - I appreciate you reaching out again :)

1. I've last replied 10 days ago, so the 14 days didn't pass yet. It would've been a good measure to share this "deadline" from the get go instead of pushing it in the last second this way :) This makes it much harder to cut out time calmly to work on this and makes it a needlessly stressful situation.
2. "Unfortunately", September 1st is the beginning of the new school year for children in Israel and my child just started a new kindergarden in a new city which takes up most of my time along with my day-job. I don't usually commit to deadlines this way (that are not my employer's) but happy to try and oblige if we set something reasonable (a week, if that's ok?).

If it's not possible and the only measure is removing the project we can do that, but that's a bit unfortunate.

Thanks!

[justice-adams-apple]

@freak4pc

from the get go instead of pushing it in the last second this way

Very valid point 👍 My mistake for not being clear, I was attempting to give you a heads up. In any case, we won't remove the project. No rush. That "rule" is typically in place for when authors don't respond to our notices in any way.

We are more then happy to wait until you have the time. I'll reach out again in a couple weeks if I haven't heard anything and we can go from there

[freak4pc]

I really appreciate it. Thank you!

[justice-adams-apple]

Hi @freak4pc 👋 , just checking in to inquire about the status of the RxSwift revision update

[freak4pc]

@justice-adams-apple see https://github.com/apple/swift-source-compat-suite/pull/723