search

swiftlang / swift

The Swift Programming Language
https://swift.org
Apache License 2.0
67.2k stars 10.32k forks source link

[SR-3698] Ensure dry run works for SourceKit #46283

Open alblue opened 7 years ago

alblue commented 7 years ago
Previous ID SR-3698
Radar None
Original Reporter @alblue
Type Sub-task
Additional Detail from JIRA | | | |------------------|-----------------| |Votes | 1 | |Component/s | Source Tooling | |Labels | Sub-task | |Assignee | None | |Priority | Medium | md5: 24ff9aa4a9de834c434f5fcd1101bacf

Parent-Task:

Issue Description:

The current build script rules for SourceKit do not respect the dry-run flag, owing to the existence of the need to make the libdispatch library. Modify this so that dry run does not build the script.

swift-ci commented 7 years ago

Comment by Félix Fischer (JIRA)

I can't reproduce this. This is my output of running ./swift/utils/build-script --dry-run

+ mkdir -p /home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert
+ env HOST_VARIABLE_linux_x86_64__SWIFT_BENCHMARK_TARGETS= HOST_VARIABLE_linux_x86_64__SWIFT_RUN_BENCHMARK_TARGETS= HOST_VARIABLE_linux_x86_64__SWIFT_SDKS=LINUX HOST_VARIABLE_linux_x86_64__SWIFT_STDLIB_TARGETS=swift-test-stdlib-linux-x86_64 HOST_VARIABLE_linux_x86_64__SWIFT_TEST_TARGETS= /home/felix/Documents/SwiftAdd2/swift/utils/build-script-impl --workspace /home/felix/Documents/SwiftAdd2 --build-dir /home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert --install-prefix /usr --host-target linux-x86_64 --stdlib-deployment-targets linux-x86_64 --host-cc /usr/bin/clang --host-cxx /usr/bin/clang++ --darwin-xcrun-toolchain default --darwin-deployment-version-osx=10.9 --darwin-deployment-version-ios=7.0 --darwin-deployment-version-tvos=9.0 --darwin-deployment-version-watchos=2.0 --cmake /usr/local/bin/cmake --cmark-build-type Debug --llvm-build-type Debug --swift-build-type Debug --swift-stdlib-build-type Debug --lldb-build-type Debug --foundation-build-type Debug --libdispatch-build-type Debug --libicu-build-type Debug --xctest-build-type Debug --swiftpm-build-type Debug --swift-enable-assertions true --swift-stdlib-enable-assertions true --swift-analyze-code-coverage false --cmake-generator Ninja --build-jobs 2 '--common-cmake-options=-G Ninja -DCMAKE_C_COMPILER:PATH=/usr/bin/clang -DCMAKE_CXX_COMPILER:PATH=/usr/bin/clang++ -DLLVM_VERSION_MAJOR:STRING=4 -DLLVM_VERSION_MINOR:STRING=0 -DLLVM_VERSION_PATCH:STRING=0 -DCLANG_VERSION_MAJOR:STRING=4 -DCLANG_VERSION_MINOR:STRING=0 -DCLANG_VERSION_PATCH:STRING=0 -DCMAKE_MAKE_PROGRAM=/usr/bin/ninja' --build-args=-j2 --cmark-cmake-options= '--llvm-cmake-options=-DLLVM_ENABLE_ASSERTIONS=TRUE -DLLVM_TARGETS_TO_BUILD=X86;ARM;AArch64;PowerPC;SystemZ' '--swift-cmake-options=-DSWIFT_STDLIB_ENABLE_SIL_OWNERSHIP=FALSE -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE -DSWIFT_FORCE_OPTIMIZED_TYPECHECKER=FALSE' --build-stdlib-deployment-targets all --ninja-bin=/usr/bin/ninja --skip-build-foundation --skip-build-xctest --skip-build-lldb --skip-build-llbuild --skip-build-libdispatch --skip-build-libicu --skip-build-swiftpm --skip-build-playgroundlogger --skip-build-playgroundsupport --build-swift-dynamic-stdlib --build-swift-dynamic-sdk-overlay --skip-build-ios-device --skip-build-ios-simulator --skip-build-tvos-device --skip-build-tvos-simulator --skip-build-watchos-device --skip-build-watchos-simulator --skip-build-android --skip-test-swift --skip-test-cmark --skip-test-lldb --skip-test-llbuild --skip-test-swiftpm --skip-test-xctest --skip-test-foundation --skip-test-libdispatch --skip-test-libicu --skip-test-playgroundlogger --skip-test-playgroundsupport --skip-test-linux --skip-test-freebsd --skip-test-cygwin --skip-test-osx --skip-test-ios-host --skip-test-ios-simulator --skip-test-tvos-host --skip-test-tvos-simulator --skip-test-watchos-host --skip-test-watchos-simulator --skip-test-android-host --skip-test-benchmarks --skip-test-optimized --android-deploy-device-path /data/local/tmp --dry-run --llvm-lit-args=-sv
Building the standard library for: swift-test-stdlib-linux-x86_64
cmark: using gold linker
+ mkdir -p /home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/cmark-linux-x86_64
+ pushd /home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/cmark-linux-x86_64
+ env /usr/local/bin/cmake -G Ninja -DCMAKE_C_COMPILER:PATH=/usr/bin/clang -DCMAKE_CXX_COMPILER:PATH=/usr/bin/clang++ -DLLVM_VERSION_MAJOR:STRING=4 -DLLVM_VERSION_MINOR:STRING=0 -DLLVM_VERSION_PATCH:STRING=0 -DCLANG_VERSION_MAJOR:STRING=4 -DCLANG_VERSION_MINOR:STRING=0 -DCLANG_VERSION_PATCH:STRING=0 -DCMAKE_MAKE_PROGRAM=/usr/bin/ninja -DCMAKE_EXE_LINKER_FLAGS:STRING=-fuse-ld=gold -DCMAKE_SHARED_LINKER_FLAGS:STRING=-fuse-ld=gold -DCMAKE_BUILD_TYPE:STRING=Debug /home/felix/Documents/SwiftAdd2/cmark
+ popd
+ /usr/local/bin/cmake --build /home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/cmark-linux-x86_64 -- -j2 all
llvm: using gold linker
+ mkdir -p /home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/llvm-linux-x86_64
+ pushd /home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/llvm-linux-x86_64
+ env /usr/local/bin/cmake -G Ninja -DCMAKE_C_COMPILER:PATH=/usr/bin/clang -DCMAKE_CXX_COMPILER:PATH=/usr/bin/clang++ -DLLVM_VERSION_MAJOR:STRING=4 -DLLVM_VERSION_MINOR:STRING=0 -DLLVM_VERSION_PATCH:STRING=0 -DCLANG_VERSION_MAJOR:STRING=4 -DCLANG_VERSION_MINOR:STRING=0 -DCLANG_VERSION_PATCH:STRING=0 -DCMAKE_MAKE_PROGRAM=/usr/bin/ninja -DCMAKE_EXE_LINKER_FLAGS:STRING=-fuse-ld=gold -DCMAKE_SHARED_LINKER_FLAGS:STRING=-fuse-ld=gold -DLLVM_ENABLE_ASSERTIONS=TRUE '-DLLVM_TARGETS_TO_BUILD=X86;ARM;AArch64;PowerPC;SystemZ' '-DCMAKE_C_FLAGS= -g' '-DCMAKE_CXX_FLAGS= -g' '-DCMAKE_C_FLAGS_RELWITHDEBINFO=-O2 -DNDEBUG' '-DCMAKE_CXX_FLAGS_RELWITHDEBINFO=-O2 -DNDEBUG' -DCMAKE_BUILD_TYPE:STRING=Debug -DLLVM_TOOL_SWIFT_BUILD:BOOL=NO -DLLVM_INCLUDE_DOCS:BOOL=TRUE -DLLVM_ENABLE_LTO:STRING= -DLLVM_TOOL_COMPILER_RT_BUILD:BOOL=TRUE -DLLVM_BUILD_EXTERNAL_COMPILER_RT:BOOL=TRUE -DLLVM_LIT_ARGS=-sv -DCMAKE_INSTALL_PREFIX:PATH=/usr/ -DINTERNAL_INSTALL_PREFIX=local /home/felix/Documents/SwiftAdd2/llvm
+ popd
symlinking the system headers (/usr/include/c++) into the local clang build directory (/home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/llvm-linux-x86_64/include).
+ ln -s -f /usr/include/c++ /home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/llvm-linux-x86_64/include
+ /usr/local/bin/cmake --build /home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/llvm-linux-x86_64 -- -j2 all
swift: using gold linker
+ mkdir -p /home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/swift-linux-x86_64
+ pushd /home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/swift-linux-x86_64
+ env /usr/local/bin/cmake -G Ninja -DCMAKE_C_COMPILER:PATH=/usr/bin/clang -DCMAKE_CXX_COMPILER:PATH=/usr/bin/clang++ -DLLVM_VERSION_MAJOR:STRING=4 -DLLVM_VERSION_MINOR:STRING=0 -DLLVM_VERSION_PATCH:STRING=0 -DCLANG_VERSION_MAJOR:STRING=4 -DCLANG_VERSION_MINOR:STRING=0 -DCLANG_VERSION_PATCH:STRING=0 -DCMAKE_MAKE_PROGRAM=/usr/bin/ninja -DSWIFT_STDLIB_ENABLE_SIL_OWNERSHIP=FALSE -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE -DSWIFT_FORCE_OPTIMIZED_TYPECHECKER=FALSE -DSWIFT_ENABLE_GOLD_LINKER=TRUE -DCMAKE_C_FLAGS= -DCMAKE_CXX_FLAGS= '-DCMAKE_C_FLAGS_RELWITHDEBINFO=-O2 -DNDEBUG' '-DCMAKE_CXX_FLAGS_RELWITHDEBINFO=-O2 -DNDEBUG' -DCMAKE_BUILD_TYPE:STRING=Debug -DLLVM_ENABLE_ASSERTIONS:BOOL=TRUE -DSWIFT_ANALYZE_CODE_COVERAGE:STRING=FALSE -DSWIFT_STDLIB_BUILD_TYPE:STRING=Debug -DSWIFT_STDLIB_ASSERTIONS:BOOL=TRUE -DSWIFT_STDLIB_ENABLE_RESILIENCE:BOOL=FALSE -DSWIFT_RUNTIME_ENABLE_COW_EXISTENTIALS:BOOL=TRUE -DSWIFT_STDLIB_USE_NONATOMIC_RC:BOOL=FALSE -DSWIFT_STDLIB_SIL_SERIALIZE_ALL:BOOL=TRUE -DSWIFT_NATIVE_LLVM_TOOLS_PATH:STRING= -DSWIFT_NATIVE_CLANG_TOOLS_PATH:STRING= -DSWIFT_NATIVE_SWIFT_TOOLS_PATH:STRING= -DSWIFT_INCLUDE_TOOLS:BOOL=TRUE -DSWIFT_BUILD_REMOTE_MIRROR:BOOL=TRUE -DSWIFT_SERIALIZE_STDLIB_UNITTEST:BOOL=FALSE -DSWIFT_STDLIB_SIL_DEBUGGING:BOOL=FALSE -DSWIFT_CHECK_INCREMENTAL_COMPILATION:BOOL=FALSE -DSWIFT_BUILD_DYNAMIC_STDLIB:BOOL=TRUE -DSWIFT_BUILD_STATIC_STDLIB:BOOL=FALSE -DSWIFT_BUILD_DYNAMIC_SDK_OVERLAY:BOOL=TRUE -DSWIFT_BUILD_STATIC_SDK_OVERLAY:BOOL=FALSE -DSWIFT_BUILD_PERF_TESTSUITE:BOOL=TRUE -DSWIFT_BUILD_EXAMPLES:BOOL=TRUE -DSWIFT_INCLUDE_TESTS:BOOL=TRUE -DSWIFT_INSTALL_COMPONENTS:STRING= -DSWIFT_EMBED_BITCODE_SECTION:BOOL=FALSE -DSWIFT_TOOLS_ENABLE_LTO:STRING= -DSWIFT_BUILD_RUNTIME_WITH_HOST_COMPILER:BOOL=FALSE -DSWIFT_HOST_VARIANT=linux -DSWIFT_HOST_VARIANT_SDK=LINUX -DSWIFT_HOST_VARIANT_ARCH=x86_64 -DLLVM_LIT_ARGS=-sv -DCOVERAGE_DB= -DSWIFT_SOURCEKIT_USE_INPROC_LIBRARY:BOOL=TRUE -DSWIFT_DARWIN_XCRUN_TOOLCHAIN:STRING=default -DSWIFT_AST_VERIFIER:BOOL=TRUE -DSWIFT_SIL_VERIFY_ALL:BOOL=FALSE -DSWIFT_RUNTIME_ENABLE_LEAK_CHECKER:BOOL=FALSE -DCMAKE_INSTALL_PREFIX:PATH=/usr/ -DSWIFT_PATH_TO_CLANG_SOURCE:PATH=/home/felix/Documents/SwiftAdd2/llvm/tools/clang -DSWIFT_PATH_TO_CLANG_BUILD:PATH=/home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/llvm-linux-x86_64 -DSWIFT_PATH_TO_LLVM_SOURCE:PATH=/home/felix/Documents/SwiftAdd2/llvm -DSWIFT_PATH_TO_LLVM_BUILD:PATH=/home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/llvm-linux-x86_64 -DSWIFT_PATH_TO_CMARK_SOURCE:PATH=/home/felix/Documents/SwiftAdd2/cmark -DSWIFT_PATH_TO_CMARK_BUILD:PATH=/home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/cmark-linux-x86_64 -DSWIFT_PATH_TO_LIBDISPATCH_SOURCE:PATH=/home/felix/Documents/SwiftAdd2/swift-corelibs-libdispatch -DSWIFT_PATH_TO_LIBDISPATCH_BUILD:PATH=/home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/libdispatch-linux-x86_64 -DSWIFT_CMARK_LIBRARY_DIR:PATH=/home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/cmark-linux-x86_64/src -DSWIFT_SDKS:STRING=LINUX -DSWIFT_EXEC:STRING=/home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/swift-linux-x86_64/bin/swiftc /home/felix/Documents/SwiftAdd2/swift
+ popd
+ /usr/local/bin/cmake --build /home/felix/Documents/SwiftAdd2/build/Ninja-DebugAssert/swift-linux-x86_64 -- -j2 all swift-test-stdlib-linux-x86_64
alblue commented 7 years ago

The problem is that the 'dry run' flag is ignored by the part of the build that handles source kit, not that any errors are seen. If you look at the full logs you'll find that the 'make' command is run for the foundation libraries even when dry run is specified (which it shouldn't). That's because the dependency for libdispatch is added and built in the script

https://github.com/apple/swift/blob/master/tools/SourceKit/CMakeLists.txt#L507-L524

Have a look for 'Generating libdispatch.so' in the logs, and the output surrounding that ...

swift-ci commented 7 years ago

Comment by Félix Fischer (JIRA)

— If you look at the full logs

Which logs do you mean?

Also... in the bug description, it says:

— Modify this so that dry run does not build the script.

I'm confused. Does it build a script? Or were you talking about it building the actual libdispatch binary?

Cheers 🙂