search

lhmouse / mcfgthread

Cornerstone of the MOST efficient std::thread on Windows for mingw-w64
https://gcc-mcf.lhmouse.com/
Other
277 stars 28 forks source link

Suggestion: Add Github Actions to compile mcfgthread. #50

Closed eko5624 closed 2 years ago

eko5624 commented 2 years ago

Thanks for your great job. I need mingw-w64-ucrt-x86_64-gcc-mcf as toolchain to compile mpv. So I download it from https://gcc-mcf.lhmouse.com/ But mingw-w64-gcc-mcf_20211231_11.2.1_x64-ucrt didn't work because of this change. I know the latest patch should fix it. Newer mingw-w64-ucrt-x86_64-gcc-mcf hasn't been released yet. So I want to compile it by myself. I don't know how to write CI.yml . So adding Github Actions will be great, thanks again.

lhmouse commented 2 years ago

Thanks for your interest!

I maintain these packages (crt-git, headers-git, gcc-mcf, gdb-mcf) as regular MSYS2 packages, so if you would like to build them yourself, here's instructions:

  1. Install development packages with pacman -S base-devel. This is required for all MSYS2 packages.
  2. Start the MSYS2 MSYS shell. Do not mistake the MINGW or UCRT shells.
  3. git clone https://github.com/lhmouse/MINGW-packages
  4. cd MINGW-packages/mingw-w64-headers-git
  5. MINGW_ARCH="mingw64 mingw32 ucrt64" makepkg-mingw I build three archs, but it looks like you only need ucrt64.
  6. If makepkg prompts for missing dependencies, install them with pacman -S and retry 5.
  7. Install the built package with pacman -U *.tar.*
  8. Do 4-7 for mingw-w64-crt-git.

headers-git and crt-git are located in this repo: https://github.com/lhmouse/MINGW-packages mcfgthread-git, gcc-mcf and gdb-mcf are located in another repo: https://github.com/lhmouse/MINGW-packages-dev

A package has to be built and installed before building the next one, strictly in this order.

lhmouse commented 2 years ago

The CI for mcfgthread predates GitHub Actions, which is defined here: https://github.com/lhmouse/mcfgthread/blob/master/.travis.yml

eko5624 commented 2 years ago

I write a workflow CI.yml. But it can't run sucessfully. Any help would be great. Please take a look. Thanks! This is error log . https://github.com/eko5624/mcfgthread/runs/4724611659?check_suite_focus=true

I'm not a programmer. Some questions confuse me~

  1. Where're the installed files using "pacman -U" ? I need to upload them, and download them to use in somewhere else.
  2. I only need mingw-w64-ucrt-x86_64-gcc-mcf(https://gcc-mcf.lhmouse.com/mingw-w64-gcc-mcf_20211231_11.2.1_x64-ucrt_bc8249a4d66c72b1641b95849914681ebe87e507.7z). The CI.yml I wrote not optimized, any suggestion would be great.
lhmouse commented 2 years ago

I write a workflow CI.yml. But it can't run sucessfully. Any help would be great. Please take a look. Thanks! This is error log . https://github.com/eko5624/mcfgthread/runs/4724611659?check_suite_focus=true

Git requires some user information, which however need not be real. Try adding

  git config --global user.email "example@example.com"
  git config --global user.name "GitHub Actions"

before cloning.

I'm not a programmer. Some questions confuse me~

  1. Where're the installed files using "pacman -U" ? I need to upload them, and download them to use in somewhere else.

Built packages are ready to be distributed and installed with pacman -U. My locally built packages have names like mingw-w64-x86_64-mcfgthread-git-r526.c8bb9f7-1-any.pkg.tar.zst which should always contain .pkg.tar. Packages are usually compressed, so they may end with .pkg.tar.xz (compressed with xz) or .pkg.tar.zst (compressed with zstd).

However, if you would like to get an out-of-the-box archive like mine, you should install those packages first. This unpacks all files to /mingw32, /mingw64, /ucrt64 etc. Those directories are complete systems and can be packed with any archivers (I use 7z personally), such that you may unpack them and get back the same enviroments.

  1. I only need mingw-w64-ucrt-x86_64-gcc-mcf(https://gcc-mcf.lhmouse.com/mingw-w64-gcc-mcf_20211231_11.2.1_x64-ucrt_bc8249a4d66c72b1641b95849914681ebe87e507.7z). The CI.yml I wrote not optimized, any suggestion would be great.

That's a complete build of GCC, not just mcfgthread. I have no experience in building packages with GitHub Actions. You might get some help from MSYS2 developers: https://gitter.im/msys2/msys2

eko5624 commented 2 years ago

@lhmouse Thank you for your helps. I try to compile mingw-w64-gcc-git（with gcc-11.2）. But no sucess. My repo is here. The errors like this:

==> Starting prepare()... .git/rebase-apply/patch:104: space before tab in indent. free (installdir); .git/rebase-apply/patch:106: trailing whitespace. }
.git/rebase-apply/patch:578: space before tab in indent. set_werrno; .git/rebase-apply/patch:589: space before tab in indent. orig_prefix[orig_prefix_len-1] = '\0'; .git/rebase-apply/patch:590: space before tab in indent. orig_prefix_len--; warning: squelched 5 whitespace errors warning: 10 lines add whitespace errors. Applying: Relocate libintl .git/rebase-apply/patch:39: indent with spaces. { .git/rebase-apply/patch:40: indent with spaces. FindClose (h); .git/rebase-apply/patch:41: indent with spaces. if ((finddata.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) && .git/rebase-apply/patch:42: indent with spaces. (finddata.dwReserved0 == IO_REPARSE_TAG_SYMLINK)) .git/rebase-apply/patch:43: indent with spaces. buf->st_mode = S_IFLNK; warning: squelched 20 whitespace errors warning: 25 lines add whitespace errors. Applying: Windows: Follow Posix dir-exists semantics more closely Applying: Windows: Use '/' not '\' in progpath and leave case as-is Applying: Windows: Don't ignore native system header dir error: patch failed: gcc/config/i386/mingw32.h:140 error: gcc/config/i386/mingw32.h: patch does not apply Applying: master Windows: New feature to allow overriding -lmsvcrt hint: Use 'git am --show-current-patch=diff' to see the failed patch Patch failed at 0001 master Windows: New feature to allow overriding -lmsvcrt When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort". ==> ERROR: A failure occurred in prepare(). Aborting... Error: Process completed with exit code 4.

I edited the PKGBUILD like this, I'm not sure if it's right.

lhmouse commented 2 years ago

The directory tree should look like follows:

mingw-w64-gcc-git
+--- gcc                    # bare clone from GCC source
+--- pkg                    # temporary directory for building packages
+--- src
|    +--- gcc               # non-bare clone; working tree for building GCC
|    +--- 0006-Windows-New-feature-to-allow-overriding.patch
|
+--- PKGBUILD
+--- 0006-Windows-New-feature-to-allow-overriding.patch

This means a patch no longer applies, either because it is applied to a wrong branch, or because upstream code has changed. The error was about Applying: master Windows: New feature to allow overriding -lmsvcrt, so the quick fix is removing the offending operation from PKGBUILD, usually in prepare():

  ${GIT_AM} ${srcdir}/0006-Windows-New-feature-to-allow-overriding.patch

After that, you must run git am --abort in gcc/gcc to cancel the failed operation, otherwise the repostory would stay unusable.

The other way to rectify the patch to resolve this error is as follows:

  1. The conflict happened in gcc/gcc, so you should cd gcc/gcc.
  2. Use git am --abort to cancel the failed am operation.
  3. Try applying the patch with git am -3 ../..0006-Windows-New-feature-to-allow-overriding.patch. Note the -3 option.
  4. Run git status -s. This prints the status of all files. Conflicts are marked by a red U in the list.
  5. Edit those files and resolve conflicts, which are marked by <<<<<<< (seven less-than symbols).
  6. Mark files resolved with git add <file>.
  7. Run git am --continue to update the commit. If there are still conflicts, go to 4.
  8. Export the updated commit with git format-patch -1 HEAD. This creates a patch file in the current directory.
  9. Move the patch back with mv <patch> ../..0006-Windows-New-feature-to-allow-overriding.patch.
  10. Update the PKGBUILD file: cd ../..; updpkgsums.
  11. Restart the build.
eko5624 commented 2 years ago

I try to compile gcc-mcf again, but it didn't work. Here is the log. Please take a look, thanks. https://github.com/eko5624/mcfgthread/runs/4814506264?check_suite_focus=true

lhmouse commented 2 years ago

Looks like a binutils bug (https://stackoverflow.com/questions/45077846/collect2-exe-error-ld-returned-5-exit-status). There might be something wrong with the GitHub Action build environment, however I am not sure how to resolve this.

lhmouse commented 2 years ago

The SO answers mentioned Windows XP, so I suspect in GitHub Actions the build task ran out of the 2GiB RAM limit. I have never seen 'exit status 5' issues because I build on a 64-bit system with 64GiB RAM.

nyfair commented 2 years ago

I confirm that there has no problem when built in a real PC. Maybe the compilation process triggers free account's github action limitation ··· libtool: install: warning: remember to run `libtool --finish /ucrt64/lib/../lib' make[4]: Nothing to be done for 'install-data-am'. make[4]: Leaving directory '/d/mcfgthread-main/mingw-w64-gcc-git/src/build-x86_64-w64-mingw32/x86_64-w64-mingw32/libatomic' make[3]: Leaving directory '/d/mcfgthread-main/mingw-w64-gcc-git/src/build-x86_64-w64-mingw32/x86_64-w64-mingw32/libatomic' make[2]: Leaving directory '/d/mcfgthread-main/mingw-w64-gcc-git/src/build-x86_64-w64-mingw32/x86_64-w64-mingw32/libatomic' make[1]: Leaving directory '/d/mcfgthread-main/mingw-w64-gcc-git/src/build-x86_64-w64-mingw32' ==> Starting package_mingw-w64-ucrt-x86_64-gcc-mcf()... ==> Tidying install... -> Removing empty directories... -> Removing libtool files... -> Purging unwanted files... -> Stripping unneeded symbols from binaries and libraries... -> Compressing man and info pages... ==> Checking for packaging issues... ==> Creating package "mingw-w64-ucrt-x86_64-gcc-mcf"... -> Generating .PKGINFO file... -> Generating .BUILDINFO file... -> Generating .MTREE file... -> Compressing package... ==> Starting package_mingw-w64-ucrt-x86_64-gcc-mcf-libs()... ==> Tidying install... -> Removing empty directories... -> Removing libtool files... -> Purging unwanted files... -> Stripping unneeded symbols from binaries and libraries... -> Compressing man and info pages... ==> Checking for packaging issues... ==> Creating package "mingw-w64-ucrt-x86_64-gcc-mcf-libs"... -> Generating .PKGINFO file... -> Generating .BUILDINFO file... -> Generating .MTREE file... -> Compressing package... ==> Finished making: mingw-w64-gcc-mcf 11.2.1.d20220118.c11.g03ccfc8b9d2-1 (Tue Jan 18 17:43:43 2022) /d/m/mingw-w64-gcc-git ❯❯❯ ls 0002-Relocate-libintl.patch 0003-Windows-Follow-Posix-dir-exists-semantics-more-close.patch 0004-Windows-Use-not-in-progpath-and-leave-case-as-is.patch 0005-Windows-Don-t-ignore-native-system-header-dir.patch 0006-Windows-New-feature-to-allow-overriding.patch 0007-Build-EXTRA_GNATTOOLS-for-Ada.patch 0008-Prettify-linking-no-undefined.patch 0010-Fix-using-large-PCH.patch 0011-Enable-shared-gnat-implib.patch 0014-gcc-9-branch-clone_function_name_1-Retain-any-stdcall-suffix.patch 0016-Ensure-lmsvcrt-precede-lkernel32.patch 0020-libgomp-Don-t-hard-code-MS-printf-attributes.patch 0021-gcc-Add-ll-and-L-length-modifiers-for-ms_printf.patch 0150-gcc-10.2.0-libgcc-ldflags.patch 9000-gcc-11-branch-Added-mcf-thread-model-support-from-mcfgthread.patch PKGBUILD PKGBUILD-dev gcc mingw-w64-ucrt-x86_64-gcc-mcf-11.2.1.d20220118.c11.g03ccfc8b9d2-1-any.pkg.tar.zst mingw-w64-ucrt-x86_64-gcc-mcf-libs-11.2.1.d20220118.c11.g03ccfc8b9d2-1-any.pkg.tar.zst pkg src /d/m/mingw-w64-gcc-git ❯❯❯ pkg/mingw-w64-ucrt-x86_64-gcc-mcf/ucrt64/bin/gcc -v Using built-in specs. COLLECT_GCC=D:\mcfgthread-main\mingw-w64-gcc-git\pkg\mingw-w64-ucrt-x86_64-gcc-mcf\ucrt64\bin\gcc.exe COLLECT_LTO_WRAPPER=D:/mcfgthread-main/mingw-w64-gcc-git/pkg/mingw-w64-ucrt-x86_64-gcc-mcf/ucrt64/bin/../lib/gcc/x86_64- w64-mingw32/11.2.1/lto-wrapper.exe Target: x86_64-w64-mingw32 Configured with: ../gcc/configure --prefix=/ucrt64 --with-local-prefix=/ucrt64/local --build=x86_64-w64-mingw32 --host=x 86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/ucrt64/x86_64-w64-mingw32/include --libex ecdir=/ucrt64/lib --enable-bootstrap --with-arch=x86-64 --with-tune=ivybridge --enable-languages=c,lto,c++ --enable-shar ed --enable-static --enable-threads=mcf --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-time=yes --di sable-libstdcxx-pch --disable-libstdcxx-debug --enable-libstdcxx-filesystem-ts=yes --disable-isl-version-check --enable- lto --enable-libgomp --disable-multilib --enable-checking=release --disable-rpath --disable-win32-registry --enable-nls --disable-werror --disable-symvers --with-libiconv --with-system-zlib --with-gmp=/ucrt64 --with-mpfr=/ucrt64 --with-mpc= /ucrt64 --with-isl=/ucrt64 --with-pkgversion='GCC with MCF thread model, built by LH_Mouse.' --with-bugurl=https://githu b.com/lhmouse/MINGW-packages-dev/issues --with-gnu-as --with-gnu-ld --with-boot-ldflags='-pipe -Wl,--dynamicbase,--high- entropy-va,--nxcompat,--default-image-base-high -Wl,--disable-dynamicbase,--default-image-base-low -static-libstdc++ -st atic-libgcc' 'LDFLAGS_FOR_TARGET=-pipe -Wl,--dynamicbase,--high-entropy-va,--nxcompat,--default-image-base-high' --enabl e-linker-plugin-flags='LDFLAGS=-static-libstdc++\ -static-libgcc\ -pipe\ -Wl,--dynamicbase,--high-entropy-va,--nxcompat, --default-image-base-high\ -Wl,--stack,12582912' --disable-tls --enable-plugin Thread model: mcf Supported LTO compression algorithms: zlib zstd gcc version 11.2.1 20220118 (GCC with MCF thread model, built by LH_Mouse.) ···

lhmouse commented 2 years ago

Yes that's as expected. I suggest you build GCC with your own PC.

eko5624 commented 2 years ago

Ok, I got it. Thanks for your help.