search

Xpra-org / gtk-osx-build

Build setup to help building the Mac OS X port of GTK+
http://gtk-osx.sourceforge.net/
0 stars 2 forks source link

Add CI workflow that builds gtk-osx-build for x86_64 and arm64 #41

Closed cpatulea closed 6 months ago

cpatulea commented 6 months ago

Add CI workflow that builds gtk-osx-build for x86_64 and arm64.

The goal is to detect build breakages early, to make fixing them easier. Also to provide a clean environment to check the build, to remove variations from developer's machines or partial build trees.

The builds take around 2 hours (x86_64) and ~45 minutes (arm64), respectively.

Some parts of the build (eg. jbuild bootstrap) are fetched from an external repository at head (https://gitlab.gnome.org/GNOME/gtk-osx/raw/master/modulesets-stable/bootstrap.modules). So if the build starts failing with no apparent changes, check upstream.

See passing CI runs: https://github.com/cpatulea/gtk-osx-build/actions/runs/8255381710

totaam commented 6 months ago

Awesome, thanks!

cpatulea commented 5 months ago

Would you mind triggering a CI run: https://github.com/Xpra-org/gtk-osx-build/actions/workflows/macos.yml

just to confirm again that it passes?

(or to grant me permission to do it)

cpatulea commented 5 months ago

Sigh.. did another CI run and it failed on some gi-docgen / typogrify issue: https://github.com/cpatulea/gtk-osx-build/actions/runs/8273499988/job/22637378808

Looks like there might still be some non-determinism.

Would still be useful to get another run in your fork as well.

totaam commented 5 months ago

Looks like there might still be some non-determinism.

Yup. Not the first time.

Would you mind triggering a CI run

Done: https://github.com/Xpra-org/gtk-osx-build/actions/runs/8275910259

or to grant me permission to do it

I've sent you an invite to the project org, I believe this should grant you this permission.

cpatulea commented 5 months ago

Your CI run passed! https://github.com/Xpra-org/gtk-osx-build/actions/runs/8275910259

cpatulea commented 5 months ago

Sigh.. did another CI run and it failed on some gi-docgen / typogrify issue: https://github.com/cpatulea/gtk-osx-build/actions/runs/8273499988/job/22637378808

Looking into this, looks like there's an earlier error in pipenv in that build:

2024-03-14T00:29:09.2712100Z Installing dependencies from Pipfile.lock (8fd481)...
2024-03-14T00:29:14.5932900Z An error occurred while installing typogrify==2.0.7 --hash=sha256:8be4668cda434163ce229d87ca273a11922cb1614cb359970b7dc96eed13cb38! Will try again.
2024-03-14T00:29:14.5935460Z Installing initially failed dependencies...
2024-03-14T00:29:16.3540670Z [InstallError]:   File "/Users/runner/.new_local/lib/python3.11/site-packages/pipenv/cli/command.py", line 233, in install
2024-03-14T00:29:16.3637910Z [InstallError]:       retcode = do_install(
2024-03-14T00:29:16.3738810Z [InstallError]:       ^^^^^^^^^^^
2024-03-14T00:29:16.3839660Z [InstallError]:   File "/Users/runner/.new_local/lib/python3.11/site-packages/pipenv/core.py", line 2052, in do_install
2024-03-14T00:29:16.3939390Z [InstallError]:       do_init(
2024-03-14T00:29:16.3970070Z [InstallError]:   File "/Users/runner/.new_local/lib/python3.11/site-packages/pipenv/core.py", line 1304, in do_init
2024-03-14T00:29:16.4071730Z [InstallError]:       do_install_dependencies(
2024-03-14T00:29:16.4173560Z [InstallError]:   File "/Users/runner/.new_local/lib/python3.11/site-packages/pipenv/core.py", line 899, in do_install_dependencies
2024-03-14T00:29:16.4255850Z [InstallError]:       batch_install(
2024-03-14T00:29:16.4282900Z [InstallError]:   File "/Users/runner/.new_local/lib/python3.11/site-packages/pipenv/core.py", line 796, in batch_install
2024-03-14T00:29:16.4284090Z [InstallError]:       _cleanup_procs(procs, failed_deps_queue, retry=retry)
2024-03-14T00:29:16.4285480Z [InstallError]:   File "/Users/runner/.new_local/lib/python3.11/site-packages/pipenv/core.py", line 703, in _cleanup_procs
2024-03-14T00:29:16.4286890Z [InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
2024-03-14T00:29:16.4288240Z [pipenv.exceptions.InstallError]: Looking in indexes: https://pypi.python.org/simple
2024-03-14T00:29:16.4290030Z [pipenv.exceptions.InstallError]: Collecting typogrify==2.0.7 (from -r /var/folders/1k/qq3pcbf12vb6vyblh81736p40000gn/T/pipenv-bokrgwmw-requirements/pipenv-au5tpqui-requirement.txt (line 1))
2024-03-14T00:29:16.4291580Z [pipenv.exceptions.InstallError]:   Using cached typogrify-2.0.7.tar.gz (12 kB)
2024-03-14T00:29:16.4292540Z [pipenv.exceptions.InstallError]:   Preparing metadata (setup.py): started
2024-03-14T00:29:16.4293670Z [pipenv.exceptions.InstallError]:   Preparing metadata (setup.py): finished with status 'done'
2024-03-14T00:29:16.4294720Z [pipenv.exceptions.InstallError]: Building wheels for collected packages: typogrify
2024-03-14T00:29:16.4295690Z [pipenv.exceptions.InstallError]:   Building wheel for typogrify (setup.py): started
2024-03-14T00:29:16.4297130Z [pipenv.exceptions.InstallError]:   Building wheel for typogrify (setup.py): finished with status 'error'
2024-03-14T00:29:16.4298450Z [pipenv.exceptions.InstallError]:   error: subprocess-exited-with-error
2024-03-14T00:29:16.4299060Z [pipenv.exceptions.InstallError]:   
2024-03-14T00:29:16.4300480Z [pipenv.exceptions.InstallError]:   × python setup.py bdist_wheel did not run successfully.
2024-03-14T00:29:16.4301270Z [pipenv.exceptions.InstallError]:   │ exit code: 1
2024-03-14T00:29:16.4301910Z [pipenv.exceptions.InstallError]:   ╰─> [42 lines of output]
2024-03-14T00:29:16.4303500Z [pipenv.exceptions.InstallError]:       Traceback (most recent call last):
2024-03-14T00:29:16.4304210Z [pipenv.exceptions.InstallError]:         File "<string>", line 2, in <module>
2024-03-14T00:29:16.4305140Z [pipenv.exceptions.InstallError]:         File "<pip-setuptools-caller>", line 34, in <module>
2024-03-14T00:29:16.4306520Z [pipenv.exceptions.InstallError]:         File "/private/var/folders/1k/qq3pcbf12vb6vyblh81736p40000gn/T/pip-install-3ww9fx2g/typogrify_18926913f44049abadb7209c8ec72c39/setup.py", line 5, in <module>
2024-03-14T00:29:16.4307570Z [pipenv.exceptions.InstallError]:           setup(
2024-03-14T00:29:16.4309210Z [pipenv.exceptions.InstallError]:         File "/Users/runner/.new_local/share/virtualenvs/etc-jUmrpTpB/lib/python3.11/site-packages/setuptools/__init__.py", line 103, in setup
2024-03-14T00:29:16.4310210Z [pipenv.exceptions.InstallError]:           return distutils.core.setup(**attrs)
2024-03-14T00:29:16.4310950Z [pipenv.exceptions.InstallError]:                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-14T00:29:16.4312130Z [pipenv.exceptions.InstallError]:         File "/Users/runner/.new_local/share/virtualenvs/etc-jUmrpTpB/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 172, in setup
2024-03-14T00:29:16.4313200Z [pipenv.exceptions.InstallError]:           ok = dist.parse_command_line()
2024-03-14T00:29:16.4314170Z [pipenv.exceptions.InstallError]:                ^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-14T00:29:16.4315400Z [pipenv.exceptions.InstallError]:         File "/Users/runner/.new_local/share/virtualenvs/etc-jUmrpTpB/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 475, in parse_command_line
2024-03-14T00:29:16.4316540Z [pipenv.exceptions.InstallError]:           args = self._parse_command_opts(parser, args)
2024-03-14T00:29:16.4317240Z [pipenv.exceptions.InstallError]:                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-14T00:29:16.4318350Z [pipenv.exceptions.InstallError]:         File "/Users/runner/.new_local/share/virtualenvs/etc-jUmrpTpB/lib/python3.11/site-packages/setuptools/dist.py", line 865, in _parse_command_opts
2024-03-14T00:29:16.4319420Z [pipenv.exceptions.InstallError]:           nargs = _Distribution._parse_command_opts(self, parser, args)
2024-03-14T00:29:16.4320140Z [pipenv.exceptions.InstallError]:                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-14T00:29:16.4321210Z [pipenv.exceptions.InstallError]:         File "/Users/runner/.new_local/share/virtualenvs/etc-jUmrpTpB/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 534, in _parse_command_opts
2024-03-14T00:29:16.4322280Z [pipenv.exceptions.InstallError]:           cmd_class = self.get_command_class(command)
2024-03-14T00:29:16.4322890Z [pipenv.exceptions.InstallError]:                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-14T00:29:16.4323980Z [pipenv.exceptions.InstallError]:         File "/Users/runner/.new_local/share/virtualenvs/etc-jUmrpTpB/lib/python3.11/site-packages/setuptools/dist.py", line 710, in get_command_class
2024-03-14T00:29:16.4325010Z [pipenv.exceptions.InstallError]:           self.cmdclass[command] = cmdclass = ep.load()
2024-03-14T00:29:16.4325650Z [pipenv.exceptions.InstallError]:                                               ^^^^^^^^^
2024-03-14T00:29:16.4326670Z [pipenv.exceptions.InstallError]:         File "/Users/runner/.new_local/share/pyenv/versions/3.11.7/lib/python3.11/importlib/metadata/__init__.py", line 202, in load
2024-03-14T00:29:16.4327670Z [pipenv.exceptions.InstallError]:           module = import_module(match.group('module'))
2024-03-14T00:29:16.4328440Z [pipenv.exceptions.InstallError]:                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-14T00:29:16.4329380Z [pipenv.exceptions.InstallError]:         File "/Users/runner/.new_local/share/pyenv/versions/3.11.7/lib/python3.11/importlib/__init__.py", line 126, in import_module
2024-03-14T00:29:16.4330430Z [pipenv.exceptions.InstallError]:           return _bootstrap._gcd_import(name[level:], package, level)
2024-03-14T00:29:16.4331310Z [pipenv.exceptions.InstallError]:                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-14T00:29:16.4332130Z [pipenv.exceptions.InstallError]:         File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2024-03-14T00:29:16.4332940Z [pipenv.exceptions.InstallError]:         File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2024-03-14T00:29:16.4333850Z [pipenv.exceptions.InstallError]:         File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
2024-03-14T00:29:16.4335090Z [pipenv.exceptions.InstallError]:         File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
2024-03-14T00:29:16.4336200Z [pipenv.exceptions.InstallError]:         File "<frozen importlib._bootstrap_external>", line 940, in exec_module
2024-03-14T00:29:16.4337110Z [pipenv.exceptions.InstallError]:         File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2024-03-14T00:29:16.4338320Z [pipenv.exceptions.InstallError]:         File "/Users/runner/.new_local/share/virtualenvs/etc-jUmrpTpB/lib/python3.11/site-packages/wheel/bdist_wheel.py", line 27, in <module>
2024-03-14T00:29:16.4339410Z [pipenv.exceptions.InstallError]:           from .macosx_libfile import calculate_macosx_platform_tag
2024-03-14T00:29:16.4340550Z [pipenv.exceptions.InstallError]:         File "/Users/runner/.new_local/share/virtualenvs/etc-jUmrpTpB/lib/python3.11/site-packages/wheel/macosx_libfile.py", line 43, in <module>
2024-03-14T00:29:16.4342080Z [pipenv.exceptions.InstallError]:           import ctypes
2024-03-14T00:29:16.4343140Z [pipenv.exceptions.InstallError]:         File "/Users/runner/.new_local/share/pyenv/versions/3.11.7/lib/python3.11/ctypes/__init__.py", line 8, in <module>
2024-03-14T00:29:16.4344140Z [pipenv.exceptions.InstallError]:           from _ctypes import Union, Structure, Array
2024-03-14T00:29:16.4344980Z [pipenv.exceptions.InstallError]:       ModuleNotFoundError: No module named '_ctypes'
2024-03-14T00:29:16.4345680Z [pipenv.exceptions.InstallError]:       [end of output]
2024-03-14T00:29:16.4346120Z [pipenv.exceptions.InstallError]:   
2024-03-14T00:29:16.4346900Z [pipenv.exceptions.InstallError]:   note: This error originates from a subprocess, and is likely not a problem with pip.
2024-03-14T00:29:16.4347860Z [pipenv.exceptions.InstallError]:   ERROR: Failed building wheel for typogrify
2024-03-14T00:29:16.4348570Z [pipenv.exceptions.InstallError]:   Running setup.py clean for typogrify
2024-03-14T00:29:16.4349150Z [pipenv.exceptions.InstallError]: Failed to build typogrify
2024-03-14T00:29:16.4350190Z [pipenv.exceptions.InstallError]: ERROR: Could not build wheels for typogrify, which is required to install pyproject.toml-based projects
2024-03-14T00:29:16.4351080Z ERROR: Couldn't install package: typogrify
2024-03-14T00:29:16.4351540Z  Package installation failed...

Looking into it more...

cpatulea commented 5 months ago

And there's a further upstream cause, during the Python build:

2024-03-14T00:28:23.5995850Z clang -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I_ctypes/darwin -I/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include/ffi -I./Include -I. -I/opt/homebrew/opt/readline/include -I/opt/homebrew/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/private/var/folders/1k/qq3pcbf12vb6vyblh81736p40000gn/T/python-build.20240314002650.3136/Python-3.11.7/Include -I/private/var/folders/1k/qq3pcbf12vb6vyblh81736p40000gn/T/python-build.20240314002650.3136/Python-3.11.7 -c /private/var/folders/1k/qq3pcbf12vb6vyblh81736p40000gn/T/python-build.20240314002650.3136/Python-3.11.7/Modules/_ctypes/callbacks.c -o build/temp.macosx-14.2-arm64-3.11/private/var/folders/1k/qq3pcbf12vb6vyblh81736p40000gn/T/python-build.20240314002650.3136/Python-3.11.7/Modules/_ctypes/callbacks.o -DUSING_MALLOC_CLOSURE_DOT_C=1 -DMACOSX -DHAVE_FFI_PREP_CIF_VAR=1 -DHAVE_FFI_PREP_CLOSURE_LOC=1 -DHAVE_FFI_CLOSURE_ALLOC=1
2024-03-14T00:28:23.7126080Z /private/var/folders/1k/qq3pcbf12vb6vyblh81736p40000gn/T/python-build.20240314002650.3136/Python-3.11.7/Modules/_ctypes/callbacks.c:438:18: error: call to undeclared function 'ffi_prep_closure'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
2024-03-14T00:28:23.7172330Z         result = ffi_prep_closure(p->pcl_write, &p->cif, closure_fcn, p);
2024-03-14T00:28:23.7173100Z                  ^
2024-03-14T00:28:23.7173690Z 1 error generated.
2024-03-14T00:28:24.1751690Z 
2024-03-14T00:28:24.1753070Z Failed to build these modules:
2024-03-14T00:28:24.1780620Z _ctypes

and now I see it's because my CI build included a commit for '--with-system-ffi'.

So I think it's not a problem with the main CI build, it's directly caused by my commit. So the CI build in main tree should be reliable.

cpatulea commented 5 months ago

Tried using the CI build artifact 'gtk-inst.zip' on my local machine, and it works, but needs explicit DYLD_LIBRARY_PATH:

# unzip (twice)
$ Users/runner/gtk/inst/bin/python3
dyld[67655]: Library not loaded: /Users/runner/gtk/inst/lib/libpython3.11.dylib
  Referenced from: <CEECF64D-28B5-3F10-914B-4183621F5504> /Users/catalinp/Downloads/Users/runner/gtk/inst/bin/python3
  Reason: tried: '/Users/runner/gtk/inst/lib/libpython3.11.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/gtk/inst/lib/libpython3.11.dylib' (no such file), '/Users/runner/gtk/inst/lib/libpython3.11.dylib' (no such file)
Abort trap: 6

$ DYLD_LIBRARY_PATH=Users/runner/gtk/inst/lib  Users/runner/gtk/inst/bin/python3
Python 3.11.8 (main, Mar 14 2024, 05:44:59) [Clang 15.0.0 (clang-1500.0.40.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>