search

swiftlang / swift-package-manager

The Package Manager for the Swift Programming Language
Apache License 2.0
9.73k stars 1.34k forks source link

[SR-15831] Investigate the flaky `PackageToolTests.testPluginCompilationBeforeBuilding` test #4353

Open abertelrud opened 2 years ago

abertelrud commented 2 years ago
Previous ID SR-15831
Radar rdar://problem/88547005
Original Reporter @abertelrud
Type Bug

Attachment: Download

Additional Detail from JIRA | | | |------------------|-----------------| |Votes | 0 | |Component/s | Package Manager | |Labels | Bug | |Assignee | @abertelrud | |Priority | Medium | md5: 1ba37e03879f706c808f975ac68401e6

Issue Description:

One of the new tests in SwiftPM, `PackageToolTests.testPluginCompilationBeforeBuilding`, is showing some flakiness (seemingly only in Linux).

This can be seen for example in https://ci.swift.org/job/oss-swift-package-linux-ubuntu-18_04/4695 (the log if which is attached here, in case it gets aged out on the server).

The problem seems to be that the last line of error output isn't always emitted. Instead of disabling the whole test, I'm disabling just the check for that line of output, and using this bug report to track investigating and fixing the underlying issue.

```
Test Case 'PackageToolTests.testPluginCompilationBeforeBuilding' started at 2022-02-06 13:05:51.532
/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-18_04/swiftpm/Tests/CommandsTests/PackageToolTests.swift:2412: error: PackageToolTests.testPluginCompilationBeforeBuilding : XCTAssertTrue failed - unexpected failure matching 'Compiling plugin MyBuildToolPlugin...
Compiling plugin MyCommandPlugin...
' against pattern contains("MyCommandPlugin/plugin.swift:7:19: error: consecutive statements on a line must be separated by ';'")
Test Case 'PackageToolTests.testPluginCompilationBeforeBuilding' failed (6.53 seconds)
```

abertelrud commented 2 years ago

@swift-cicreate