build: slightly improve the build for libdispatch

[compnerd]

Avoid polluting the build directory a small amount given that we can use -fmodule-map-file= for the C/C++ build of libdispatch. Unfortunately, for the Swift build, we need to have the file copied over due to the umbrella header resolution.

Hopefully this reduces some of the race conditions that we have seen in the build.

Thanks to @dgregor for reminding me of the flag!

[etcwilde]

CC @rokhinip, this is the source of our little race condition in CI that crops up from time to time.

[compnerd]

That VFS idea :thinking:

[compnerd]

@swift-ci please test

[etcwilde]

Good. I like it. No more copying things around in the source directories during the build.

[DougGregor]

@swift-ci please test

[etcwilde]

Linux failure:

/home/build-user/swift-corelibs-foundation/Sources/Foundation/DispatchData+DataProtocol.swift:14:8: error: cannot load underlying module for 'Dispatch'
import Dispatch
       ^

Windows failure:

CMake Error at dispatch/cmake_install.cmake:59 (file):
  file INSTALL cannot find
  "C:/Users/swift-ci/jenkins/workspace/swift-corelibs-libdispatch-PR-windows/swift-corelibs-libdispatch/dispatch/module.modulemap":
  File exists.
Call Stack (most recent call first):
  cmake_install.cmake:42 (include)

[etcwilde]

Hmm, Linux VFS overlay not quite pointing in the right place maybe?

[DougGregor]

Hmm. The Foundation build has include paths pointing into the Dispatch source directory:

-I /home/build-user/build/buildbot_linux/libdispatch-linux-x86_64/src/swift/swift -I /home/build-user/build/buildbot_linux/libdispatch-linux-x86_64 -I /home/build-user/swift-corelibs-libdispatch -I /home/build-user/swift-corelibs-libdispatch/src -I /home/build-user/build/buildbot_linux/libdispatch-linux-x86_64/src -I /home/build-user/swift-corelibs-libdispatch/src/BlocksRuntime

So we either need to also add the VFS flags to the Foundation build, or we need to first install Dispatch and then switch the Foundation build to use the installed Dispatch.

[etcwilde]

switch the Foundation build to use the installed Dispatch.

That would be a day for celebration.

[compnerd]

@DougGregor correct, I had missed that I had given the wrong visibility to the flags. This is now resolved and -vfsoverlay should propagate properly when building against the build tree. I think that the now current state should build both against an uninstalled as well as installed dispatch. If you use -D dispatch_DIR=... you will prefer the build tree and if you do not, it will prefer the "system version".

[compnerd]

@swift-ci please test

[etcwilde]

For tracking purposes:

• rdar://92756762
• rdar://94289755
• rdar://105782611

[DougGregor]

@DougGregor correct, I had missed that I had given the wrong visibility to the flags. This is now resolved and -vfsoverlay should propagate properly when building against the build tree. I think that the now current state should build both against an uninstalled as well as installed dispatch. If you use -D dispatch_DIR=... you will prefer the build tree and if you do not, it will prefer the "system version".

Sounds like exactly what we want. 🤞

[DougGregor]

The Linux build got much farther! Now the XCTest tests are failing with

# command stderr:
<unknown>:0: error: cannot load underlying module for 'Dispatch'

[DougGregor]

@swift-ci please test Windows

[DougGregor]

Windows got to the same point; now it's the XCTest failures, but Foundation built and tested properly (!)

[compnerd]

Ah, nice! The XCTest failures are not surprising. Fortunately, I had recently enhanced that path for Windows - we just need to make sure that we are passing along the dispatch flags to the tests, not just the build.

[DougGregor]

So exciting!

[compnerd]

Please test with the following PRs: https://github.com/apple/swift-corelibs-xctest/pull/437

@swift-ci please test