search

shinchiro / mpv-winbuild-cmake

CMake-based MinGW-w64 Cross Toolchain
616 stars 114 forks source link

Ubuntu Docker fails during install step for 'cppwinrt' #712

Open twilson90 opened 5 days ago

twilson90 commented 5 days ago 
[build 10/11] RUN ninja gcc:
0.480 [1/94] Creating directories for 'mingw-w64-headers'
0.489 [2/94] Creating directories for 'gcc-binutils'
0.489 [3/94] Creating directories for 'gcc-wrapper'
0.490 [4/94] Creating directories for 'mingw-w64'
0.490 [5/94] Creating directories for 'cppwinrt'
0.490 [6/94] Creating directories for 'gcc'
0.491 [7/94] Creating directories for 'mingw-w64-crt'
0.491 [8/94] Creating directories for 'gendef'
0.492 [9/94] Creating directories for 'rustup'
0.492 [10/94] Creating directories for 'winpthreads'
0.504 [11/94] No download step for 'rustup'
0.505 [12/94] No download step for 'mingw-w64-headers'
0.505 [13/94] No download step for 'gcc-wrapper'
0.505 [14/94] No download step for 'gendef'
0.506 [15/94] No download step for 'winpthreads'
0.506 [16/94] No download step for 'mingw-w64-crt'
0.519 [17/94] No update step for 'gcc-wrapper'
0.519 [18/94] No update step for 'rustup'
0.532 [19/94] No patch step for 'gcc-wrapper'
0.533 [20/94] No patch step for 'rustup'
0.545 [21/94] No configure step for 'gcc-wrapper'
0.648 [22/94] Performing build step for 'gcc-wrapper'
0.661 [23/94] Setting up target directories and symlinks
0.676 [24/94] Deleting build directory of gcc-wrapper package after install
0.695 [25/94] Completed 'gcc-wrapper'
3.287 [26/94] Performing download step (git clone) for 'cppwinrt'
3.302 [27/94] No update step for 'cppwinrt'
3.307 [28/94] Performing write-head step for 'cppwinrt'
3.321 [29/94] No patch step for 'cppwinrt'
3.533 [30/94] Performing configure step for 'cppwinrt'
22.88 [31/94] Performing download step (git clone) for 'mingw-w64'
22.89 [32/94] No update step for 'mingw-w64'
22.90 [33/94] Performing write-head step for 'mingw-w64'
22.91 [34/94] No patch step for 'mingw-w64'
22.93 [35/94] No configure step for 'mingw-w64'
22.94 [36/94] No build step for 'mingw-w64'
22.96 [37/94] No install step for 'mingw-w64'
22.98 [38/94] Completed 'mingw-w64'
22.99 [39/94] No update step for 'gendef'
23.01 [40/94] No patch step for 'gendef'
24.38 [41/94] Performing configure step for 'gendef'
24.79 [42/94] Performing build step for 'gendef'
24.83 [43/94] Performing install step for 'gendef'
24.85 [44/94] Deleting build directory of gendef package after install
24.87 [45/94] Completed 'gendef'
27.68 [46/94] Performing build step for 'cppwinrt'
27.72 [47/94] Performing install step for 'cppwinrt'
27.72 FAILED: toolchain/cppwinrt-prefix/src/cppwinrt-stamp/cppwinrt-install /opt/build64/toolchain/cppwinrt-prefix/src/cppwinrt-stamp/cppwinrt-install
27.72 cd /opt/build64/toolchain/cppwinrt-prefix/src/cppwinrt-build && /usr/bin/cmake -P /opt/build64/toolchain/cppwinrt-prefix/src/cppwinrt-stamp/cppwinrt-install-.cmake && /usr/bin/cmake -E touch /opt/build64/toolchain/cppwinrt-prefix/src/cppwinrt-stamp/cppwinrt-install
27.72 CMake Error at /opt/build64/toolchain/cppwinrt-prefix/src/cppwinrt-stamp/cppwinrt-install-.cmake:49 (message):
27.72   Command failed: 1
27.72
27.72    '/usr/bin/cmake' '-Dmake=' '-Dconfig=' '-P' '/opt/build64/toolchain/cppwinrt-prefix/src/cppwinrt-stamp/cppwinrt-install--impl.cmake'
27.72
27.72   See also
27.72
27.72     /opt/build64/toolchain/cppwinrt-prefix/src/cppwinrt-stamp/cppwinrt-install-*.log
27.72
27.72
49.22 [48/94] Performing download step (download, verify and extract) for 'gcc-binutils'
57.92 [49/94] Performing download step (download, verify and extract) for 'gcc'
67.51 [50/94] Performing configure step for 'rustup'
67.51 ninja: build stopped: subcommand failed.
------
Dockerfile.windows:59
--------------------
  57 |     WORKDIR /opt/build64
  58 |     RUN ninja download
  59 | >>> RUN ninja gcc
  60 |     RUN ninja mpv
  61 |
--------------------
ERROR: failed to solve: process "/bin/sh -c ninja gcc" did not complete successfully: exit code: 1

Dockerfile:

FROM ubuntu:24.04 AS build

RUN apt update -y
RUN apt install -y build-essential checkinstall bison flex gettext git mercurial subversion ninja-build gyp cmake yasm nasm automake pkgconf libtool libtool-bin gcc-multilib g++-multilib clang lld libc++1 libc++abi1 libgmp-dev libmpfr-dev libmpc-dev libgcrypt-dev gperf ragel texinfo autopoint re2c asciidoc python3-pip docbook2x unzip p7zip-full curl libmimalloc-dev ccache
ENV PIP_BREAK_SYSTEM_PACKAGES=1
RUN pip3 install rst2pdf meson mako jsonschema

WORKDIR /opt
RUN git clone https://github.com/shinchiro/mpv-winbuild-cmake.git

RUN cmake -DTARGET_ARCH=x86_64-w64-mingw32 \
    -DGCC_ARCH=x86-64-v3 \
    -DSINGLE_SOURCE_LOCATION="/home/user/packages" \
    -DRUSTUP_LOCATION="/home/user/install_rustup" \
    -G Ninja -B build64 -S mpv-winbuild-cmake

WORKDIR /opt/build64
RUN ninja download
RUN ninja gcc
RUN ninja mpv

FROM scratch AS artifact
COPY --from=build /opt/mpv/out /

FROM build AS release

Not having much luck here!

Andarwinux commented 5 days ago

/opt/build64/toolchain/cppwinrt-prefix/src/cppwinrt-stamp/cppwinrt-install-*.log plz

I guess it's just a network problem.

twilson90 commented 5 days ago

It's not producing any logs:

Docker_Desktop_vSQ0eNikfX

I don't think it's a network problem, I tried to build in docker container a month ago and had the same issue.

Andarwinux commented 5 days ago

So maybe it's because wget doesn't exist? I'm not sure if these dependencies automatically pull wget in.

twilson90 commented 4 days ago

That solved it! Thank you.

I'm almost there but I'm running into an error at the last hurdle. It's failing to build harfbuzz and/or libjxl, but the logs don't appear very helpful.

docker-build-log.txt harfbuzz-build-out.log libjxl-build-out.log

Andarwinux commented 4 days ago

This issue is caused by OOM.

For historical reasons, the parallel build mechanism of this script is suboptimal: there is no “global thread pool.”

For example, on a 16-core system, Ninja performs 16 external projects simultaneously by default, and each external project tells Ninja/Make to emit 16 instances of the compiler, which in the worst case can result in 256 instances. This is fine for Clang, because Clang emits "cc1" in-process. However, GCC's driver and cc1 are separate executables, which can lead to hundreds of processes being created simultaneously and immediately exhaust all RAM.

As far as I know, Ninja does not yet support the GNU Make jobserver protocol, so we cannot fundamentally fix this issue. However, you can switch to Clang, or limit the top-level Ninja parallelism with -j, for example, ninja -j2 mpv, or add -DCPU_COUNT=2 in your CMake setup to restrict the number of concurrent compilers.

twilson90 commented 4 days ago

Thank you! With your help I've finally managed to 'successfully' build a windows executable. Some problems though:

Andarwinux commented 4 days ago
  • When I try running it in Windows it runs for a moment before exiting. Maybe the mpv master branch is currently having issues? Nothing helpful in the run-time log: log.txt

You need to add keep-open=yes to keep mpv running without any input.

  • I want to build my own fork of mpv. I tried replacing /home/user/packages/mpv with my own but the build fails: mpv-build-out.log

This is a GCC 14 false positive, you can add -Wno-error=incompatible-pointer-types at https://github.com/shinchiro/mpv-winbuild-cmake/blob/9c153bced077742a874e4ed61c7a33ca90e92f17/packages/mpv.cmake#L56

To replace it with a different source, just replace GIT_REPOSITORY, and if your branch name is not master, specify GIT_TAG and GIT_REMOTE_NAME.

https://github.com/shinchiro/mpv-winbuild-cmake/blob/9c153bced077742a874e4ed61c7a33ca90e92f17/packages/mpv.cmake#L25

  • Also where do I specify mpv build options? (stuff like -Dmanpage-build=disabled)

Just add it after CONFIGURE_COMMAND.

twilson90 commented 3 days ago

I've tried: -Dc_args='-Wno-error=int-conversion -Wno-error=incompatible-pointer-types' and "-Dc_args='-Wno-error=int-conversion -Wno-error=incompatible-pointer-types'"

but it's failing to run the configure command.

There's an error log generated that I haven't been able to look at due to issues with Docker. Presumably though I'm just not setting the argument correctly.

(EDIT : my mistake, I had included an extra feature in the configure command without installing the prerequsites and this was causing it to fail)

Also the previous successful build was closing immediately on opening any media file, here's another log (no errors): log.txt

Andarwinux commented 3 days ago

It looks like there was a segmentation fault and you need to follow the instructions here to get the backtrace.

If you are using clang to build mpv, you can use windbg to do this step in a simpler way via GUI.

twilson90 commented 2 days ago

Here's the backtrace from drmingw:

drmingw.txt

Thanks so much for your continung assistance.

Andarwinux commented 2 days ago 
mpv.exe caused an Access Violation at location 00007FF62055A15B in module mpv.exe Reading from location FFFFFFFFFFFFFFFF.

AddrPC           Params
00007FF62055A15B 0000000000000000 000000000000000B 0000000000000000  mpv.exe!mpv_hook_continue+0x7cadb
00007FF62055EAFC 000077DD802E21F4 0000000000002F24 0000000000000002  mpv.exe!mpv_hook_continue+0x8147c
00007FF62055FBED 000002A600000000 0000000000000000 0000000000000000  mpv.exe!mpv_hook_continue+0x8256d
00007FF620561A17 000002A6F907C230 0000000000000000 0000000000000000  mpv.exe!mpv_hook_continue+0x84397
00007FF8D3AE9333 0000000000000000 0000000000000000 0000000000000000  ucrtbase.dll!thread_start<unsigned int (__cdecl*)(void *),1>+0x93
00007FF8D411257D 0000000000000000 0000000000000000 0000000000000000  KERNEL32.DLL!BaseThreadInitThunk+0x1d
00007FF8D61EAF08 0000000000000000 0000000000000000 0000000000000000  ntdll.dll!RtlUserThreadStart+0x28

Can you reproduce this issue with a prebuilt mpv (here or upstream CI)? This looks like an mpv bug, if it can be reproduced using the CI version, please open an issue upstream.

twilson90 commented 1 day ago

I finally managed to build a working version by using -DGCC_ARCH=x86-64 instead of -DGCC_ARCH=x86-64-v3

Or possibly by running ninja -j2 mpv instead of ninja -j4 mpv

Andarwinux commented 1 day ago

A v3 build obviously requires a v3-capable CPU to run. By the way, GCC has serious flaws with AVX2/AVX512 CodeGen for Windows targets, while Clang is fine.