[SR-13635] CMark Test fails

[swift-ci]

Previous ID	SR-13635
Radar	None
Original Reporter	pushpinderpalsingh (JIRA User)
Type	Bug

Environment

master branch, macOS Big Sur Beta, Xcode 12

Additional Detail from JIRA

| | | |------------------|-----------------| |Votes | 2 | |Component/s | | |Labels | Bug | |Assignee | None | |Priority | Medium | md5: e0f5c41f8df8d01a23a4aebd07ad1f5e

relates to:

• SR-14724 --skip-test-cmark does not skip CTest

Issue Description:

I am trying to compile Swift for a while now. The compilation goes perfectly but some test case fails which results in the process to terminate with an exit status of 65.
I am using Xcode 12 ( not beta ) and macOS Big Sur Beta. I am using this command to build the Swift compiler.

{{utils/build-script --skip-build-benchmarks }}
--skip-ios --skip-watchos --skip-tvos --swift-darwin-supported-archs "x86_64"
--cmake-c-launcher="$(which sccache)" --cmake-cxx-launcher="(which sccache)"
--release-debuginfo --test
--xcode

And getting the following output (I am just adding the failing test cases)

=== BUILD AGGREGATE TARGET RUN_TESTS OF PROJECT cmark WITH CONFIGURATION RelWithDebInfo ===

Check dependencies

PhaseScriptExecution CMake\ PostBuild\ Rules /Volumes/ExtremeSSD/swift-project/build/Xcode-RelWithDebInfoAssert/cmark-macosx-x86_64/cmark.build/RelWithDebInfo/RUN_TESTS.build/Script-209180F299BC48E5B78D44FC.sh
cd /Volumes/ExtremeSSD/swift-project/cmark
/bin/sh -c /Volumes/ExtremeSSD/swift-project/build/Xcode-RelWithDebInfoAssert/cmark-macosx-x86_64/cmark.build/RelWithDebInfo/RUN_TESTS.build/Script-209180F299BC48E5B78D44FC.sh
/usr/local/Cellar/cmake/3.18.3/bin/ctest --force-new-ctest-process -C RelWithDebInfo
Test project /Volumes/ExtremeSSD/swift-project/build/Xcode-RelWithDebInfoAssert/cmark-macosx-x86_64
Start 1: api_test
1/9 Test #​1: api_test ......................... Passed 0.02 sec
Start 2: html_normalization
2/9 Test #​2: html_normalization ............... Passed 0.94 sec
Start 3: spectest_library
3/9 Test #​3: spectest_library .................***Failed 0.11 sec
Start 4: pathological_tests_library
4/9 Test #​4: pathological_tests_library .......***Failed 0.09 sec
Start 5: roundtriptest_library
5/9 Test #​5: roundtriptest_library ............***Failed 0.08 sec
Start 6: entity_library
6/9 Test #​6: entity_library ...................***Failed 0.06 sec
Start 7: spectest_executable
7/9 Test #​7: spectest_executable ..............***Failed 0.08 sec
Start 8: smartpuncttest_executable
8/9 Test #​8: smartpuncttest_executable ........***Failed 0.07 sec
Start 9: regressiontest_executable
9/9 Test #​9: regressiontest_executable ........***Failed 0.07 sec

22% tests passed, 7 tests failed out of 9

Total Test time (real) = 1.53 sec

The following tests FAILED:
3 - spectest_library (Failed)
4 - pathological_tests_library (Failed)
5 - roundtriptest_library (Failed)
6 - entity_library (Failed)
7 - spectest_executable (Failed)
8 - smartpuncttest_executable (Failed)
9 - regressiontest_executable (Failed)
Errors while running CTest
make: *** [RUN_TESTS_buildpart_0] Error 8
Command /bin/sh failed with exit code 2

=== BUILD AGGREGATE TARGET ZERO_CHECK OF PROJECT cmark WITH CONFIGURATION RelWithDebInfo ===

Check dependencies

PhaseScriptExecution CMake\ Rules /Volumes/ExtremeSSD/swift-project/build/Xcode-RelWithDebInfoAssert/cmark-macosx-x86_64/cmark.build/RelWithDebInfo/ZERO_CHECK.build/Script-8A9EE83BA8FD420BB7903C9C.sh
cd /Volumes/ExtremeSSD/swift-project/cmark
/bin/sh -c /Volumes/ExtremeSSD/swift-project/build/Xcode-RelWithDebInfoAssert/cmark-macosx-x86_64/cmark.build/RelWithDebInfo/ZERO_CHECK.build/Script-8A9EE83BA8FD420BB7903C9C.sh
echo ""

make -f /Volumes/ExtremeSSD/swift-project/build/Xcode-RelWithDebInfoAssert/cmark-macosx-x86_64/CMakeScripts/ReRunCMake.make
make[1]: `/Volumes/ExtremeSSD/swift-project/build/Xcode-RelWithDebInfoAssert/cmark-macosx-x86_64/CMakeFiles/cmake.check_cache' is up to date.

• • BUILD FAILED **

The following build commands failed:
PhaseScriptExecution CMake\ PostBuild\ Rules /Volumes/ExtremeSSD/swift-project/build/Xcode-RelWithDebInfoAssert/cmark-macosx-x86_64/cmark.build/RelWithDebInfo/RUN_TESTS.build/Script-209180F299BC48E5B78D44FC.sh
(1 failure)
ERROR: command terminated with a non-zero exit status 65, aborting

Additional configuration information (utils/update-checkout --dump-hashes):

{

"https-clone-pattern": "https://github.com/%s.git", 

"ssh-clone-pattern": "git@github.com:%s.git", 

"repos": {

    "sourcekit-lsp": {

        "remote": {

            "id": "apple/sourcekit-lsp"

        }

    }, 

    "indexstore-db": {

        "remote": {

            "id": "apple/indexstore-db"

        }

    }, 

    "llbuild": {

        "remote": {

            "id": "apple/swift-llbuild"

        }

    }, 

    "swift-xcode-playground-support": {

        "remote": {

            "id": "apple/swift-xcode-playground-support"

        }

    }, 

    "swift-syntax": {

        "remote": {

            "id": "apple/swift-syntax"

        }

    }, 

    "swift-corelibs-xctest": {

        "remote": {

            "id": "apple/swift-corelibs-xctest"

        }

    }, 

    "swiftpm": {

        "remote": {

            "id": "apple/swift-package-manager"

        }

    }, 

    "swift-tools-support-core": {

        "remote": {

            "id": "apple/swift-tools-support-core"

        }

    }, 

    "cmake": {

        "platforms": \[

            "Linux"

        \], 

        "remote": {

            "id": "KitWare/CMake"

        }

    }, 

    "swift-driver": {

        "remote": {

            "id": "apple/swift-driver"

        }

    }, 

    "yams": {

        "remote": {

            "id": "jpsim/Yams"

        }

    }, 

    "icu": {

        "platforms": \[

            "Linux"

        \], 

        "remote": {

            "id": "unicode-org/icu"

        }

    }, 

    "swift-corelibs-foundation": {

        "remote": {

            "id": "apple/swift-corelibs-foundation"

        }

    }, 

    "llvm-project": {

        "remote": {

            "id": "apple/llvm-project"

        }

    }, 

    "swift-argument-parser": {

        "remote": {

            "id": "apple/swift-argument-parser"

        }

    }, 

    "ninja": {

        "remote": {

            "id": "ninja-build/ninja"

        }

    }, 

    "swift-integration-tests": {

        "remote": {

            "id": "apple/swift-integration-tests"

        }

    }, 

    "swift": {

        "remote": {

            "id": "apple/swift"

        }

    }, 

    "swift-stress-tester": {

        "remote": {

            "id": "apple/swift-stress-tester"

        }

    }, 

    "cmark": {

        "remote": {

            "id": "apple/swift-cmark"

        }

    }, 

    "swift-format": {

        "remote": {

            "id": "apple/swift-format"

        }

    }, 

    "swift-corelibs-libdispatch": {

        "remote": {

            "id": "apple/swift-corelibs-libdispatch"

        }

    }

}, 

"branch-schemes": {

    "repro": {

        "repos": {

            "sourcekit-lsp": "241744da9ba73ab9951e7fb70a75e04779642302", 

            "swift-syntax": "94fc5ae3f34fac87380756b9c17ea7c6752a227b", 

            "llbuild": "3f236e84ebf77fbc4a1c6a41500ffa6b6709180a", 

            "swift-xcode-playground-support": "88043d7d320f92598efb39408c3f4b1903a4fff6", 

            "indexstore-db": "b7cc40f64a72ce1368b9b2f7bcfa9d9950698e0b", 

            "swift-corelibs-xctest": "d728b6456d453be212038bd8fe01ceebf5b7ce4f", 

            "swiftpm": "726eff2c35fc3620bc0afde6befecc9b0246b0b5", 

            "swift-tools-support-core": "cf5db868eeed2f41ddf1d7ac9b202b5738d67dc1", 

            "cmake": "skip", 

            "swift-driver": "5a7b75cac10aeee72f89cb965754167739dcd497", 

            "yams": "81a65c4069c28011ee432f2858ba0de49b086677", 

            "icu": "skip", 

            "swift-corelibs-foundation": "e9d8347e64ece3de453d617887af7229f73358d8", 

            "llvm-project": "40349c774225a0989b6a70bd16122b7d486c1dfb", 

            "swift-argument-parser": "15351c1cd009eba0b6e438bfef55ea9847a8dc4a", 

            "ninja": "a1f879b29c9aafe6a2bc0ba885701f8f4f19f772", 

            "swift-integration-tests": "3ceb61d33ea8885938eace0add830d143a3059cd", 

            "swift": "44c8052bb53108464a43ac8d8f3297e487439d7d", 

            "swift-stress-tester": "4d8f8a790d32a3ced310c6b68c46ca8e6f71f52c", 

            "cmark": "9c8096a23f44794bde297452d87c455fc4f76d42", 

            "swift-format": "d4bba6e22891ff1813e8267e36f2b00307684366", 

            "swift-corelibs-libdispatch": "f13ea5dcc055e5d2d7c02e90d8c9907ca9dc72e1"

        }, 

        "aliases": \[

            "repro"

        \]

    }

}

}%

[vguerra]

I found the very same issue using Xcode Beta 12.2 beta (12B5018i)

[swift-ci]

Comment by Roland Lariotte (JIRA)

I am experiencing the same issue with:

macOS Catalina 10.15.7
Xcode 12.0.1 (12A7300)

[vguerra]

I found the reason for the failure of the Cmark tests ( at least in debug builds ), see my post in the forums.

Not sure how to fix it though! :/

[swift-ci]

Comment by Ryan Aveo (JIRA)

Also experiencing on Xcode 12.2 beta and Catalina 10.15.6

[dabrahams]

I observe the same thing building with Xcode:

https://gist.github.com/dabrahams/6f3840c0609ad7c30736a5aca378d269

[dabrahams]

You can work around this by adding --skip-test-cmark to the build-script invocation.

[WowbaggersLiquidLunch]

--skip-test-cmark doesn't seem to work for me now. It used to work before.

Build script output: https://gist.github.com/WowbaggersLiquidLunch/c274d73173e8740f5861b479f998b788

[swift-ci]

Comment by Oscar Swanros (JIRA)

For anyone still having trouble with this, removing the tests with --skip-test-cmark seems to do the trick.

[swift-ci]

Comment by Roberto Gomes Rosmaninho Neto (JIRA)

I got the same error at MacOS 12.3 and XCode 13.3.0.0.1.1645755326
Build Script output: SwiftBuildingError.md