search

swiftlang / swift

The Swift Programming Language
https://swift.org
Apache License 2.0
67.5k stars 10.35k forks source link

[SR-4608] Missing dependencies in build system? #47185

Open dabrahams opened 7 years ago

dabrahams commented 7 years ago
Previous ID SR-4608
Radar rdar://problem/31648833
Original Reporter @dabrahams
Type Bug
Additional Detail from JIRA | | | |------------------|-----------------| |Votes | 0 | |Component/s | Project Infrastructure | |Labels | Bug, BuildScript | |Assignee | @bob-wilson | |Priority | Medium | md5: 816665ab1842c12203bf934db4311a18

Issue Description:

I keep seeing things like this when I make changes in the Standard library, especially to types with lots of @_transparent methods:

: && /Users/Shared/dabrahams/s/build/Ninja-Release/llvm-macosx-x86_64/./bin/clang++  -fno-stack-protector -stdlib=libc++ -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -Werror=date-time -std=c++11 -fcolor-diagnostics -Wdocumentation -Wimplicit-fallthrough -Wunreachable-code -Woverloaded-virtual -DOBJC_OLD_DISPATCH_PROTOTYPES=0 -fno-sanitize=all -DLLVM_DISABLE_ABI_BREAKING_CHECKS_ENFORCING=1 -O3 -DNDEBUG -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk -dynamiclib -Wl,-headerpad_max_install_names -stdlib=libc++   -target x86_64-apple-macosx10.9 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk -arch x86_64 -F /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/../../../Developer/Library/Frameworks -mmacosx-version-min=10.9 -Wl,-sectcreate,__TEXT,__info_plist,/Users/Shared/dabrahams/s/build/Ninja-Release/swift-macosx-x86_64/stdlib/public/SwiftOnoneSupport/Info.plist -Wl,-application_extension  -L/Users/Shared/dabrahams/s/build/Ninja-Release/swift-macosx-x86_64/./lib/swift/macosx/x86_64 -L/Users/Shared/dabrahams/s/build/Ninja-Release/swift-macosx-x86_64/./bin/../lib/swift/macosx/x86_64 -L/Users/Shared/dabrahams/s/build/Ninja-Release/swift-macosx-x86_64/./bin/../lib/swift/macosx -o lib/swift/macosx/x86_64/libswiftSwiftOnoneSupport.dylib -install_name @rpath/libswiftSwiftOnoneSupport.dylib stdlib/public/SwiftOnoneSupport/macosx/x86_64/SwiftOnoneSupport.o -L/Users/Shared/dabrahams/s/build/Ninja-Release/llvm-macosx-x86_64/./lib lib/swift/macosx/x86_64/libswiftCore.dylib -framework Foundation -framework CoreFoundation && :
Undefined symbols for architecture x86_64:
  "__T0SS7ContentV5_packs22PackedUnsignedIntegersVys6UInt60Vs6UInt16VGSgx5utf16_AI14bitsPerElementts10CollectionRzAI8Iterator_0K0RTzlFZs07LazyMapL0VyxAIGSRys5UInt8VGcfU_A2IcfU_", referenced from:
      __T0SS7ContentV5_packs22PackedUnsignedIntegersVys6UInt60Vs6UInt16VGSgx5utf16_AI14bitsPerElementts10CollectionRzAI8Iterator_0K0RTzlFZs07LazyMapL0VyxAIGSRys5UInt8VGcfU_A2IcfU_TA in SwiftOnoneSupport.o

simd.o is another one that was failing to be linked recently.

Is it possible these .o files aren't dependent on the standard library's .o file in the build system?

dabrahams commented 7 years ago

@swift-ci create

dabrahams commented 7 years ago

In fact I very commonly see overlays and other components that depend on the standard library building long before the standard library is finished compiling. Is it possible they're all depending on the .swiftmodule but not the .dylib, but they need to depend on both?

Please make this a priority; it really gets in the way of standard library development.

/cc najacque (JIRA User)

dabrahams commented 7 years ago

najacque (JIRA User) Can you please escalate this? It's gotten to the point that I need to do

cd ~/src/s/swift && find . -name '*.swift*' -not -path './stdlib/public/core/*' -print0 | xargs -0 touch

after I change anything in the standard library. Otherwise I get link errors and my benchmark results are out-of-date.

@shahmishal You might need to add something like the above to any incremental builds in CI until this is fixed.

dabrahams commented 7 years ago

zisko (JIRA User) I think you told me you were going to add a patch to the build script to squash this problem until we could implement a real solution?

dabrahams commented 7 years ago

Escalating to @bob-wilson since nothing seems to be happening here.

Bob, this problem makes it impossible to be sure you are getting accurate benchmark or test results unless you touch all the other swift files whenever the stdlib (and presumably, the compiler too?) changes. IMO this is an intolerable fragility. It didn't use to be this way; it regressed.

I've been manually using this, but it's still easy to forget:

#!/usr/bin/env bash -e -o pipefail
top=$(git rev-parse --show-toplevel 2>/dev/null || echo "$HOME/src/s/swift")
cd "${top}"
find . -name '*.swift*' -not -path './stdlib/public/core/*' -not -path './test/*' -not -path './validation-test/*' -print0 | xargs -0 touch
dabrahams commented 7 years ago

Weird; I just saw this happen when I built without the extra "touch" step:

+ /Users/dave/brew/bin/cmake --build /Users/dave/src/s/build/Ninja-Release/swift-macosx-x86_64 -- -j20 all swift-test-stdlib-macosx-x86_64 swift-benchmark-macosx-x86_64
[1/306] Generating SwiftRevision.inc
[2/84] Generating Substring.swift from Substring.swift.gyb with ptr size = 8
...

It's as though generating SwiftRevision.inc brought a bunch of targets up-to-date or something.