add a dockerfile for building CI for redhat UBI

[tomerd]

motivation: support redhat distro

changes: add a dockerfile for building a toolchain on redhat UBI 8

[tomerd]

@swift-ci test

[tomerd]

@swift-ci test

[tomerd]

@shahmishal can you make sure I can trigger CI here

[shahmishal]

@shahmishal can you make sure I can trigger CI here

You have access as the admin of this repo, however the CI system is having issues from starting the job.

[tomerd]

@swift-ci test

[shahmishal]

@swift-ci test

[shahmishal]

Failure on aarch64:

FAILED: unittests/runtime/SwiftRuntimeTests 
: && /home/build-user/build/buildbot_linux/llvm-linux-aarch64/./bin/clang++ -Wno-unknown-warning-option -Werror=unguarded-availability-new -fno-stack-protector -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-class-memaccess -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -fdiagnostics-color -ffunction-sections -fdata-sections -Werror=switch -Wdocumentation -Wimplicit-fallthrough -Wunreachable-code -Woverloaded-virtual -DOBJC_OLD_DISPATCH_PROTOTYPES=0 -O3 -DNDEBUG -Wl,--gc-sections  -fuse-ld=gold stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/__/CompatibilityOverride/CompatibilityOverride.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/AnyHashableSupport.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/Array.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/AutoDiffSupport.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/Bincompat.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/Casting.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/CrashReporter.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/Demangle.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/DynamicCast.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/Enum.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/EnvironmentVariables.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ErrorObjectCommon.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ErrorObjectNative.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/Errors.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ErrorDefaultImpls.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/Exclusivity.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ExistentialContainer.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/Float16Support.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/FoundationSupport.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/FunctionReplacement.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/Heap.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/HeapObject.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ImageInspectionCommon.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ImageInspectionMachO.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ImageInspectionELF.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ImageInspectionCOFF.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ImageInspectionStatic.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ImageInspectionWasm.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/KeyPaths.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/KnownMetadata.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/Metadata.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/MetadataLookup.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/Numeric.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/Once.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/Portability.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ProtocolConformance.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/RefCount.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ReflectionMirror.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/RuntimeInvocationsTracking.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/SwiftDtoa.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/SwiftTLSContext.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ThreadingError.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/AccessibleFunction.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ErrorObject.mm.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/SwiftObject.mm.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/SwiftValue.mm.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ReflectionMirrorObjC.mm.o stdlib/public/runtime/CMakeFiles/swiftRuntime-linux-aarch64.dir/ObjCRuntimeGetImageNameFromClass.mm.o stdlib/public/LLVMSupport/CMakeFiles/swiftLLVMSupport-linux-aarch64.dir/ErrorHandling.cpp.o stdlib/public/LLVMSupport/CMakeFiles/swiftLLVMSupport-linux-aarch64.dir/Hashing.cpp.o stdlib/public/LLVMSupport/CMakeFiles/swiftLLVMSupport-linux-aarch64.dir/MemAlloc.cpp.o stdlib/public/LLVMSupport/CMakeFiles/swiftLLVMSupport-linux-aarch64.dir/SmallPtrSet.cpp.o stdlib/public/LLVMSupport/CMakeFiles/swiftLLVMSupport-linux-aarch64.dir/SmallVector.cpp.o stdlib/public/LLVMSupport/CMakeFiles/swiftLLVMSupport-linux-aarch64.dir/StringRef.cpp.o stdlib/public/CMakeFiles/swiftDemangling-linux-aarch64.dir/__/__/lib/Demangling/Context.cpp.o stdlib/public/CMakeFiles/swiftDemangling-linux-aarch64.dir/__/__/lib/Demangling/Demangler.cpp.o stdlib/public/CMakeFiles/swiftDemangling-linux-aarch64.dir/__/__/lib/Demangling/ManglingUtils.cpp.o stdlib/public/CMakeFiles/swiftDemangling-linux-aarch64.dir/__/__/lib/Demangling/NodePrinter.cpp.o stdlib/public/CMakeFiles/swiftDemangling-linux-aarch64.dir/__/__/lib/Demangling/Punycode.cpp.o stdlib/public/CMakeFiles/swiftDemangling-linux-aarch64.dir/__/__/lib/Demangling/Remangler.cpp.o stdlib/public/CMakeFiles/swiftDemangling-linux-aarch64.dir/__/__/lib/Demangling/NodeDumper.cpp.o stdlib/public/CMakeFiles/swiftDemangling-linux-aarch64.dir/__/__/lib/Demangling/Errors.cpp.o stdlib/public/Threading/CMakeFiles/swiftThreading-linux-aarch64.dir/__/__/__/lib/Threading/C11.cpp.o stdlib/public/Threading/CMakeFiles/swiftThreading-linux-aarch64.dir/__/__/__/lib/Threading/Linux.cpp.o stdlib/public/Threading/CMakeFiles/swiftThreading-linux-aarch64.dir/__/__/__/lib/Threading/Pthreads.cpp.o stdlib/public/Threading/CMakeFiles/swiftThreading-linux-aarch64.dir/__/__/__/lib/Threading/Win32.cpp.o unittests/runtime/CMakeFiles/SwiftRuntimeTests.dir/Array.cpp.o unittests/runtime/CMakeFiles/SwiftRuntimeTests.dir/CompatibilityOverrideRuntime.cpp.o unittests/runtime/CMakeFiles/SwiftRuntimeTests.dir/CompatibilityOverrideConcurrency.cpp.o unittests/runtime/CMakeFiles/SwiftRuntimeTests.dir/Concurrent.cpp.o unittests/runtime/CMakeFiles/SwiftRuntimeTests.dir/Metadata.cpp.o unittests/runtime/CMakeFiles/SwiftRuntimeTests.dir/Enum.cpp.o unittests/runtime/CMakeFiles/SwiftRuntimeTests.dir/ExtendedExistential.cpp.o unittests/runtime/CMakeFiles/SwiftRuntimeTests.dir/Refcounting.cpp.o unittests/runtime/CMakeFiles/SwiftRuntimeTests.dir/Stdlib.cpp.o unittests/runtime/CMakeFiles/SwiftRuntimeTests.dir/StackAllocator.cpp.o unittests/runtime/CMakeFiles/SwiftRuntimeTests.dir/Actor.cpp.o unittests/runtime/CMakeFiles/SwiftRuntimeTests.dir/TaskStatus.cpp.o -o unittests/runtime/SwiftRuntimeTests -L/home/build-user/build/buildbot_linux/llvm-linux-aarch64/./lib   -L/home/build-user/build/buildbot_linux/swift-linux-aarch64/libdispatch-linux-aarch64-prefix/lib -Wl,-rpath,/home/build-user/build/buildbot_linux/llvm-linux-aarch64/./lib:/home/build-user/build/buildbot_linux/swift-linux-aarch64/libdispatch-linux-aarch64-prefix/lib:/home/build-user/build/buildbot_linux/swift-linux-aarch64/lib/swift/linux/aarch64  /home/build-user/build/buildbot_linux/llvm-linux-aarch64/lib/libLLVMSupport.a  -lpthread  /home/build-user/build/buildbot_linux/llvm-linux-aarch64/lib/libllvm_gtest_main.a  /home/build-user/build/buildbot_linux/llvm-linux-aarch64/lib/libllvm_gtest.a  -lpthread  -lBlocksRuntime  lib/swift/linux/aarch64/libswiftDistributed.so  /home/build-user/build/buildbot_linux/llvm-linux-aarch64/lib/libLLVMSupport.a  -lrt  -lpthread  -lm  /usr/lib64/libz.so  /usr/lib64/libtinfo.so  /home/build-user/build/buildbot_linux/llvm-linux-aarch64/lib/libLLVMDemangle.a  -ldl  -lpthread  lib/swift/linux/aarch64/libswift_Concurrency.so  -ldispatch  lib/swift/linux/aarch64/libswiftCore.so  -Wl,-rpath-link,/home/build-user/build/buildbot_linux/swift-linux-aarch64/lib/swift/linux/aarch64 && :
/home/build-user/build/buildbot_linux/llvm-linux-aarch64/lib/libLLVMSupport.a(Path.cpp.o):Path.cpp:function std::_Sp_counted_ptr_inplace<llvm::sys::fs::detail::DirIterState, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_get_deleter(std::type_info const&): error: undefined reference to 'std::_Sp_make_shared_tag::_S_eq(std::type_info const&)'
/home/build-user/build/buildbot_linux/llvm-linux-aarch64/lib/libLLVMSupport.a(VirtualFileSystem.cpp.o):VirtualFileSystem.cpp:function std::_Sp_counted_ptr_inplace<(anonymous namespace)::RealFSDirIter, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_get_deleter(std::type_info const&): error: undefined reference to 'std::_Sp_make_shared_tag::_S_eq(std::type_info const&)'
/home/build-user/build/buildbot_linux/llvm-linux-aarch64/lib/libllvm_gtest.a(gtest-all.cc.o):gtest-all.cc:function std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::vector(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&): error: undefined reference to 'std::__throw_bad_array_new_length()'
/home/build-user/build/buildbot_linux/llvm-linux-aarch64/lib/libllvm_gtest.a(gtest-all.cc.o):gtest-all.cc:function testing::internal::edit_distance::CalculateOptimalEdits(std::vector<unsigned long, std::allocator<unsigned long> > const&, std::vector<unsigned long, std::allocator<unsigned long> > const&): error: undefined reference to 'std::__throw_bad_array_new_length()'
/home/build-user/build/buildbot_linux/llvm-linux-aarch64/lib/libllvm_gtest.a(gtest-all.cc.o):gtest-all.cc:function testing::internal::edit_distance::CalculateOptimalEdits(std::vector<unsigned long, std::allocator<unsigned long> > const&, std::vector<unsigned long, std::allocator<unsigned long> > const&): error: undefined reference to 'std::__throw_bad_array_new_length()'
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
[102/109][ 93%][50.065s] Building CXX object unittests/SourceKit/Support/CMakeFiles/SourceKitSupportTests.dir/ImmutableTextBufferTest.cpp.o
[103/109][ 94%][50.107s] Building CXX object unittests/runtime/LongTests/CMakeFiles/SwiftRuntimeLongTests.dir/LongRefcounting.cpp.o
[104/109][ 95%][50.282s] Linking CXX executable unittests/Sema/swiftSemaTests
[105/109][ 96%][51.106s] Building CXX object unittests/SourceKit/SwiftLang/CMakeFiles/SourceKitSwiftLangTests.dir/CursorInfoTest.cpp.o

[tomerd]

@MaxDesiatov @drexin what do you think is missing here?

[drexin]

@MaxDesiatov @drexin what do you think is missing here?

Those symbols come from the C++ stdlib. Maybe we are running into some incompatibility?

[MaxDesiatov]

I'm not experiencing any issues when building LLVM, but I'm passing --libcxx=false to build-script, maybe that has enough impact? Here's my Dockerfile:

FROM redhat/ubi8

RUN yum install -y  \
  git               \
  gcc-c++           \
  clang             \
  cmake             \
  make              \
  libcurl-devel     \
  libedit-devel     \
  libuuid-devel     \
  libxml2-devel     \
  ncurses-devel     \
  python3-devel     \
  rsync             \
  sqlite-devel

WORKDIR /root/source

RUN git clone https://github.com/apple/swift && \
    git -C swift fetch  && \
    git -C swift checkout release/5.7 && \
    ./swift/utils/update-checkout --clone --scheme release/5.7 --skip-repository swift

# Run the base build without SwiftPM or Foundation
RUN ./swift/utils/build-script \
      --build-swift-static-stdlib \
      --build-swift-static-sdk-overlay \
      --build-swift-stdlib-unittest-extra \
      --release                \
      --skip-build-benchmarks  \
      --bootstrapping=off      \
      --libcxx=false

RUN cd yams && git checkout 5.0.1

RUN ./swift/utils/build-script \
      --lldb \
      --release \
      --foundation \
      --libdispatch \
      --indexstore-db \
      --sourcekit-lsp \
      --swiftdocc \
      --llbuild \
      --swiftpm \
      --swift-driver \
      --xctest \
      --libicu \
      --swiftdocc \
      --build-ninja \
      --install-all \
      --installable-package=$(pwd)/installable-package.tar.gz \
      --install-destdir=$(pwd)/installable-package

[tomerd]

the script here is pretty vanilla, what is different is that this is UBI 8

[MaxDesiatov]

--bootstrapping=off could also be the reason, I find that default bootstrapping setting causes build issues on Linux frequently for me

[MaxDesiatov]

Do we still plan to pursue UBI 8 after support for UBI 9 has been merged?

[tomerd]

I think we can do that if there is specific demand. hopefully UBI9 is enough for most folks.

[shahmishal]

@tomerd Do we still need this?

[tomerd]

I think we already added UBI9 after this was created. no need in UBI8 unless there is specific demand