search

conda-forge / qt-feedstock

A conda-smithy repository for qt.
BSD 3-Clause "New" or "Revised" License
12 stars 57 forks source link

Rebuild icu69 #214

Closed izahn closed 2 years ago

izahn commented 2 years ago

Checklist

Closes https://github.com/conda-forge/qt-feedstock/pull/207 Closes https://github.com/conda-forge/qt-feedstock/pull/212 Closes https://github.com/conda-forge/qt-feedstock/pull/213

conda-forge-linter commented 2 years ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

akrherz commented 2 years ago

This is impressive and hard work you folks are putting in, but I am curious of the goals here given what has happened with the new qt-main repo. Could somebody tersely summarize what is happening ? Perhaps I am the only one curious :)

hmaarrfk commented 2 years ago

Maybe this thread can help https://github.com/conda-forge/qt-feedstock/issues/192

I remember somebody announcing it on a thread somewhere, but I'm not sure where.

The goal here is mostly to migrate this until something else is fixed between qt and qt-main (and qt-webengine).

Ultimately, I think the qt package will just depend on qt-main and qt-webengine but I'm not sure when that will happen.

hmaarrfk commented 2 years ago

@izahn how did you hack the aarch64 build again? The hack seems to require python 2 which is not in the mian channel for aarch64.

izahn commented 2 years ago

@izahn how did you hack the aarch64 build again? The hack seems to require python 2 which is not in the mian channel for aarch64.

Sorry which hack? I might have done something but I have an old brain and don't remember.

izahn commented 2 years ago

This is impressive and hard work you folks are putting in, but I am curious of the goals here given what has happened with the new qt-main repo. Could somebody tersely summarize what is happening ? Perhaps I am the only one curious :)

The goal is to keep qt in sync with the rest of the conda-forge ecosystem by merging the icu69 migration PR. Keeping conda-forge packages in sync with the global pinning is the mechanism by which packages are kept compatible with each other. Because the icu69 migration PR has been languishing qt is currently incompatible with the most recent tensorflow-base and everything else that has been migrated to icu 69

akrherz commented 2 years ago

The goal is to keep qt in sync with the rest of the conda-forge ecosystem

Thank you and totally agree as I have been thrashing about a lot within conda-forge due to the missing qt + icu69 combination :) This would be awesome to figure out!

hmaarrfk commented 2 years ago

@izahn was it you that cared about aarch64? maybe it was somebody else. I'm trying to use the conda-forge channel for the hack instead of main: https://github.com/conda-forge/qt-feedstock/pull/213/commits/51b448515364d6c54ad36733476b8a80a9ed2e2c

Lets see if that works.

izahn commented 2 years ago

@izahn was it you that cared about aarch64? maybe it was somebody else. I'm trying to use the conda-forge channel for the hack instead of main: 51b4485

That is horrifying :-)

hmaarrfk commented 2 years ago

what makes you think we no longer need to merge build and host for qtwebengine?

izahn commented 2 years ago

what makes you think we no longer need to merge build and host for qtwebengine?

python build-locally.py completed the build without error.

hmaarrfk commented 2 years ago

oh. very cool!

hmaarrfk commented 2 years ago

Do we have time to try removing the openssl hack? I feel like openssl is already installed in the host environment....

izahn commented 2 years ago

Do we have time to try removing the openssl hack? I feel like openssl is already installed in the host environment....

My motivation for improvements here is limited because https://github.com/conda-forge/qt-main-feedstock will soon replace it if all goes well. I think we should just get the icu migration out the door and call it a day.

hmaarrfk commented 2 years ago

I guess you should at least include the fix for finding python-2 and openssl from conda-forge instead of main. Using mamba will help the solve time a bit too.

izahn commented 2 years ago

I feel optimistic that what we have here works and is good enough. I'm building linux_64_ now, any volunteers to build the others? We need

hmaarrfk commented 2 years ago

I can't seem to access the conda forge aarch server.

@Tobias-Fischer do you have an other machine?

Tobias-Fischer commented 2 years ago

No, I was building on the conda-forge aarch server ..

izahn commented 2 years ago

linux_64_ build is at https://anaconda.org/izahn/qt/5.12.9/download/linux-64/qt-5.12.9-ha98a1a1_5.tar.bz2 and the log file is at https://www.dropbox.com/s/enwa7l4oegbcdu3/qt_linux_64_.log?dl=0

Tobias-Fischer commented 2 years ago

In principle I am happy to build on aarch (this is what I promised), however 1) I think the PR should first be properly reviewed so we don't end up building multiple times, and 2) I'll be on leave till 4 January ..

izahn commented 2 years ago

@conda-forge/qt this is ready for review.

hmaarrfk commented 2 years ago

Webkit seems to be failing on aarch:

[2396/16095] CXX obj/skia/skia_opts_crc32/SkOpts_crc32.o
FAILED: obj/skia/skia_opts_crc32/SkOpts_crc32.o
$SRC_DIR/g++ -MMD -MF obj/skia/skia_opts_crc32/SkOpts_crc32.o.d -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BROWSING -DSAFE_BROWSING_CSD -DSAFE_BROWSING_DB_LOCAL -DOFFI
CIAL_BUILD -DCHROMIUM_BUILD -DFIELDTRIAL_TESTING_ENABLED -DTOOLKIT_QT -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DSK_IGNORE_LINEONLY_AA_CONVEX_PATH_OPTS -DS
K_HAS_PNG_LIBRARY -DSK_HAS_WEBP_LIBRARY -DSK_HAS_JPEG_LIBRARY -DSK_SUPPORT_GPU=1 -DSK_GPU_WORKAROUNDS_HEADER=\"gpu/config/gpu_driver_bug_workaround_autogen.h\" -DSK_GAMMA_EXPONENT=1.2 -DSK_GAMMA_CONTRAST=0.2 -DSK_DEFAULT_FONT_CACHE_
LIMIT=20971520 -Igen -I../../3rdparty/chromium -I../../3rdparty/chromium/skia/config -I../../3rdparty/chromium/skia/ext -I../../3rdparty/chromium/third_party/skia/include/c -I../../3rdparty/chromium/third_party/skia/include/config -
I../../3rdparty/chromium/third_party/skia/include/core -I../../3rdparty/chromium/third_party/skia/include/effects -I../../3rdparty/chromium/third_party/skia/include/encode -I../../3rdparty/chromium/third_party/skia/include/gpu -I../
../3rdparty/chromium/third_party/skia/include/images -I../../3rdparty/chromium/third_party/skia/include/lazy -I../../3rdparty/chromium/third_party/skia/include/pathops -I../../3rdparty/chromium/third_party/skia/include/pdf -I../../3
rdparty/chromium/third_party/skia/include/pipe -I../../3rdparty/chromium/third_party/skia/include/ports -I../../3rdparty/chromium/third_party/skia/include/utils -I../../3rdparty/chromium/third_party/skia/src/gpu -I../../3rdparty/chr
omium/third_party/skia/src/sksl -I../../3rdparty/chromium/third_party/skia/include/codec -I../../3rdparty/chromium/third_party/skia/include/private -I../../3rdparty/chromium/third_party/skia/include/client/android -I../../3rdparty/c
hromium/third_party/skia/src/codec -I../../3rdparty/chromium/third_party/skia/src/core -I../../3rdparty/chromium/third_party/skia/src/image -I../../3rdparty/chromium/third_party/skia/src/images -I../../3rdparty/chromium/third_party/
skia/src/opts -I../../3rdparty/chromium/third_party/skia/src/pdf -I../../3rdparty/chromium/third_party/skia/src/ports -I../../3rdparty/chromium/third_party/skia/src/shaders -I../../3rdparty/chromium/third_party/skia/src/shaders/grad
ients -I../../3rdparty/chromium/third_party/skia/src/sfnt -I../../3rdparty/chromium/third_party/skia/src/utils -I../../3rdparty/chromium/third_party/skia/src/lazy -I../../3rdparty/chromium/third_party/skia/third_party/gif -I../../3r
dparty/chromium/third_party/skia/src/effects/gradients -march=armv8-a+crc -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -funwind-tables -fPIC -pipe -pthread -fno-omit-frame-pointer -g0 -fvisibility=hidden -O2 -fno
-ident -fdata-sections -ffunction-sections -Wno-unused-local-typedefs -Wno-maybe-uninitialized -Wno-deprecated-declarations -fno-delete-null-pointer-checks -Wno-comments -Wno-dangling-else -Wno-packed-not-aligned -Wno-missing-field-
initializers -Wno-unused-parameter -std=gnu++14 -fno-exceptions -fno-rtti -fvisibility-inlines-hidden -Wno-narrowing -Wno-attributes -Wno-class-memaccess -Wno-subobject-linkage -c ../../3rdparty/chromium/third_party/skia/src/opts/Sk
Opts_crc32.cpp -o obj/skia/skia_opts_crc32/SkOpts_crc32.o
In file included from ../../3rdparty/chromium/third_party/skia/src/opts/SkOpts_crc32.cpp:11:
../../3rdparty/chromium/third_party/skia/src/opts/SkChecksum_opts.h:22:1: error: template with C linkage
   22 | template <typename T, typename P>
      | ^~~~~~~~
In file included from ../../3rdparty/chromium/third_party/skia/src/opts/SkChecksum_opts.h:17,
                 from ../../3rdparty/chromium/third_party/skia/src/opts/SkOpts_crc32.cpp:11:
$BUILD_PREFIX/lib/gcc/aarch64-conda-linux-gnu/9.4.0/include/arm_acle.h:33:1: note: 'extern "C"' linkage started here
   33 | extern "C" {
      | ^~~~~~~~~~
../../3rdparty/chromium/third_party/skia/src/opts/SkOpts_crc32.cpp:17:1: error: expected '}' at end of input
   17 | }
      | ^
In file included from ../../3rdparty/chromium/third_party/skia/src/opts/SkChecksum_opts.h:17,
                 from ../../3rdparty/chromium/third_party/skia/src/opts/SkOpts_crc32.cpp:11:
$BUILD_PREFIX/lib/gcc/aarch64-conda-linux-gnu/9.4.0/include/arm_acle.h:33:12: note: to match this '{'
   33 | extern "C" {
      |            ^
[2397/16095] CXX obj/skia/skia_core_and_effects/SkXfermodeImageFilter.o
ninja: build stopped: subcommand failed.
make[4]: Leaving directory '$SRC_DIR/qtwebengine/src/core'
make[3]: Leaving directory '$SRC_DIR/qtwebengine/src/core'
izahn commented 2 years ago

Hmm, looks like the CI got stuck and never got to that point: https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=429108&view=logs&jobId=6494c7a6-a019-59a3-6f3d-9bab8ca96717&j=6494c7a6-a019-59a3-6f3d-9bab8ca96717&t=47f1ac7e-d0be-5d8d-6da0-bd921ed9abfb

@conda-forge-admin please rerender

hmaarrfk commented 2 years ago

I don't think you'll get to that point on Azure. It uses qemu and is very slow.

izahn commented 2 years ago

@conda-forge/qt this is ready for review.

I was wrong, still some things to work out. Will ping you again when this is ready.

Tobias-Fischer commented 2 years ago

Possibly related:

hmaarrfk commented 2 years ago

i guess we can try to downgrade the compiler?

izahn commented 2 years ago

@hmaarrfk and @Tobias-Fischer I added you to my forked repo in case you want to push anything directly there. Or we can continue with https://github.com/conda-forge/qt-feedstock/pull/213 if that's easier. I have no preference for which PR gets merged, I just want to get this migration taken care of so I can install the latest tensorflow and qt in the same environment.

hmaarrfk commented 2 years ago

the thing is, nothing changed in terms of the compiler version for Aarch. so I'm not sure

hmaarrfk commented 2 years ago

I think that this package should be moved forward ignoring the aarch failures. This issue should be able to be dealt with separately.

The main issue, opencv + qt, will be addressed here.

For opencv, we don't build with qt support for aarch (yet) so I think it will be safe to ignore.

https://github.com/conda-forge/opencv-feedstock/blob/master/recipe/meta.yaml#L94

hmaarrfk commented 2 years ago

I'm not sure who can build osx, or windows.

Tobias-Fischer commented 2 years ago

I'm happy to give the aarch build a try - are the patches etc all fine?

hmaarrfk commented 2 years ago

All the links above refer to gcc patches. I'm not sure where to apply those in conda-forge.

Tobias-Fischer commented 2 years ago

Oh, sorry for the confusing question. I was referring to the status of the PR, making sure that everything else looks good. I'll try aarch with a downgraded compiler.

hmaarrfk commented 2 years ago

I mean, the current state compiles on linux it seems.

Tobias-Fischer commented 2 years ago

I've built the win and osx packages:

I currently don't have access to the conda-forge aarch server, and thus cannot built the aarch package. As soon as I have access, I am happy to try building the package with gcc11 (rather than 9).

Tobias-Fischer commented 2 years ago

Actually gcc 9.3 should work - it looks like the bug was introduced in gcc 9.4. The last aarch build was done with gcc 9.3.

@hmaarrfk - would you be happy to try?

hmaarrfk commented 2 years ago

@Tobias-Fischer i would be happy to trigger the build. I don't have much time to write the yml file that will pin to 9.3 for Aarch. could you do that and rerender and push? i can trigger after that.

hmaarrfk commented 2 years ago

@conda-forge-admin please rerender

hmaarrfk commented 2 years ago

I started the rebuild on aarch. the rerendering shows that the other builds (not aarch64) were not affected. So hopefully all should be ready in 1 day.

hmaarrfk commented 2 years ago

I'm not really sure how to get the environment to solve. if you can get that working with an older compiler i can launch it.

Tobias-Fischer commented 2 years ago

It seems like indeed for libgcc=9.3 and icu=69 the environment is impossible to solve. Is there any objection in pinning the compilers version to 11? In this case I can solve locally (without the additional nspr/sqlite pins).

@izahn - could you please re-invite me to your fork?

hmaarrfk commented 2 years ago

i don't think I'm against going to gcc 11.

izahn commented 2 years ago

It seems like indeed for libgcc=9.3 and icu=69 the environment is impossible to solve. Is there any objection in pinning the compilers version to 11? In this case I can solve locally (without the additional nspr/sqlite pins).

@izahn - could you please re-invite me to your fork?

@Tobias-Fischer I just sent the invite again, sorry for the delay. Happy new year!

Tobias-Fischer commented 2 years ago

Okay, I got the PR to build on aarch. At the moment the tests fail because some system requirements (libGL etc) are not installed; I asked Isuru to install them and will run the build again once that's done.

To get the PR to build, I needed to tweak some bits:

izahn commented 2 years ago

Has there been any movement here @Tobias-Fischer @hmaarrfk ? I think we just need to build and upload right?

Tobias-Fischer commented 2 years ago

Hi @isuruf - sorry to ping you. Could you please install the libGL system dependencies on the aarch server so I can build again with passing tests?

Thanks!

isuruf commented 2 years ago

Done

Tobias-Fischer commented 2 years ago

Thanks - I triggered the aarch build.

Tobias-Fischer commented 2 years ago

Aarch package: https://anaconda.org/TobiasRobotics/qt/5.12.9/download/linux-aarch64/qt-5.12.9-hb57a898_5.tar.bz2 Aarch log: https://www.dropbox.com/s/pelomc1k725go3z/build_log_aarch.txt?dl=0