search

conda-forge / qt-main-feedstock

A conda-smithy repository for qt-main.
BSD 3-Clause "New" or "Revised" License
5 stars 29 forks source link

General discussion for Qt6 migration #49

Open hmaarrfk opened 2 years ago

hmaarrfk commented 2 years ago

Comment:

There is alot of worry that Qt6 will cause a slow rollout for many libraries.

Already we can see that many of the packages have not yet completed the qt -> qt-main migration in

https://conda-forge.org/status/#qt515

Many of the packages seem abandoned, so it might be best to:

  1. Archive those packages.
  2. Assess the situation with a few key packages.
  3. Compare how other linux distros are doing it.

It seems at least that ubuntu is at least providing both:

``` $ sudo apt search libqt[56]net Sorting... Done Full Text Search... Done libqt5network5/jammy-updates,now 5.15.3+dfsg-2ubuntu0.1 amd64 [installed,automatic] Qt 5 network module libqt5networkauth5/jammy 5.15.3-1 amd64 online account access for Qt apps - Library libqt5networkauth5-dev/jammy 5.15.3-1 amd64 online account access for Qt apps - Development Files libqt6network6/jammy 6.2.4+dfsg-2ubuntu1 amd64 Qt 6 network module libqt6networkauth6/jammy 6.2.4-1 amd64 Qt 6 QtNetworkAuth library libqt6networkauth6-dev/jammy 6.2.4-1 amd64 Qt 6 QtNetworkAuth - development files ```

Other relevant discussion:

Updating Qt/PyQt to 5.15 was a huge deal of work, so on the Spyder team we won't be looking at Qt6 for at least a year, perhaps more. And I don't know who else would be willing to do this as a paid effort, sorry.

Tobias-Fischer commented 2 years ago

@scopatz - tagging you here as you seem to be the sole maintainer of quite a few packages that have not yet been migrated to qt 5.15.

ccordoba12 commented 2 years ago

Pasting here @Tobias-Fischer https://github.com/conda-forge/qt-main-feedstock/pull/44#issuecomment-1243047585:

So if we don't allow installation of qt5 and qt6 side-by-side, and are not careful in downstream feedstocks (where some maintainers might opt to manually migrate to qt6 without also still building for qt5), won't we fairly soon run into the situation where packages become incompatible with each other? We would need to maintain two branches of qt5/6-webengine etc. anyhow, regardless of the renaming.

Note that I don't have particularly strong feelings about this, I am just curious how we envision the transition to qt6 :).

This is a good point, one that I haven't considered before. Feedstocks would need to explicitly declare if they support only qt5 or qt5 and 6, in case we decide not to version Qt6. I don't know if this could be handled by the migration bot, i.e. by adding a preventive constraint on qt/pyqt/pyside < 6 for all feedstocks that currently depend on those packages.

ccordoba12 commented 2 years ago

Compare how other linux distros are doing it. It seems at least that ubuntu is at least providing both

Yeah, they do that because it can take maintainers a couple of years to move their packages from qt5 to 6, especially for C++ apps such as Paraview, as @jschueller mentioned.

hmaarrfk commented 2 years ago

My main question is:

If upstream supports it, then I am in favor of supporting it in conda-forge too.

If upstream does not support it, I dont think we should work too hard to get both installed in the same environment. It could take longer to do that than "a few years"

Tobias-Fischer commented 2 years ago

Yes, upstream supports installation of qt5 and qt6 at the same time. This happens in e.g. Ubuntu, too. See https://doc.qt.io/qt-6/packaging-recommendations.html

In #44 @jschueller already follows these packaging recommendations. I am not sure if we need to update our qt5 builds in some way, too, though?

jschueller commented 2 years ago

there should be no conflicts with the qt5 version, we install in qt6 subfolders, and for binaries only versioned symlinks are available, which means qmake would point to the qt5 version and qmake6 to the qt6 version and although conda wont allow to install different versions of the same package, it means it will be ready when the time comes

Tobias-Fischer commented 2 years ago

So if we keep qt-main as qt5, and create a new package qt6-main that contains qt6 (which is very easy), we should be good to go? I suggest an easy way to test things would be to then install qt-main and qt6-main in the same environment, and check whether both the tests for qt-main and qt6-main still work.

We also might want to try whether https://manpages.ubuntu.com/manpages/jammy/man1/qtchooser.1.html (https://code.qt.io/cgit/qtsdk/qtchooser.git/) works with conda packaging. If qtchooser works, that would be amazing!

jschueller commented 2 years ago

what do you think @ccordoba12 ? should I close #44 and request it as a new qt6-main feedstock ?

ccordoba12 commented 2 years ago

what do you think @ccordoba12 ? should I close https://github.com/conda-forge/qt-main-feedstock/pull/44 and request it as a new qt6-main feedstock ?

We have mentioned pros and cons but I think we haven't made a decision on that yet. I'm also uncertain on how to proceed. Should we call a vote on this? @hmaarrfk, what do you think?

We also might want to try whether https://manpages.ubuntu.com/manpages/jammy/man1/qtchooser.1.html (https://code.qt.io/cgit/qtsdk/qtchooser.git/) works with conda packaging. If qtchooser works, that would be amazing!

For that we're using qt.conf at the moment. So we'd probably need to patch Qt6 to point to a differently named qt.conf, so that Qt5 and 6 can coexist in the same environment.

hmaarrfk commented 2 years ago

I agree with the coinstallation attempt.

hmaarrfk commented 2 years ago

I mostly wanted to have the Convo here so as not to loose it and to clutter jschulers technical work

hmaarrfk commented 2 years ago

We can likely create a qt5-main package, and do the switchover in 1 year or two from qt5-main to qt6-main. An other mini migrator!

Tobias-Fischer commented 2 years ago

For that we're using qt.conf at the moment. So we'd probably need to patch Qt6 to point to a differently named qt.conf, so that Qt5 and 6 can coexist in the same environment.

It seems like this is simply a matter of renaming qt.conf to qt6.conf: "For this purpose, a file named qt6.conf can be used instead of the qt.conf file. If both files exist in the directory described above, qt6.conf is used." (https://doc-snapshots.qt.io/qt6-6.4/qt-conf.html)

what do you think @ccordoba12 ? should I close #44 and request it as a new qt6-main feedstock ?

There is no need to create a new feedstock - we can simply change the name of the package that is created. See e.g. https://github.com/conda-forge/libignition-gazebo-feedstock/blob/main/recipe/meta.yaml where the name consists of the base name and the major version.

jschueller commented 1 year ago

I renamed the package to qt6-main in #44 and checked it can be co-installed with qt-main and still pass its tests. For win32 unfortunately some plugins and exes still clobber with qt-main, I'll try to ask upstream. Would this be considered good enough for merging though ?

hmaarrfk commented 1 year ago

if you add an exclusive constraint on qt6-main and qt-main for now, that might be ok.

jschueller commented 1 year ago

ok, I just did that

jschueller commented 1 year ago

opened upstream issue for the coinstallation on windows: https://bugreports.qt.io/browse/QTBUG-107009 what do you think now ?

hmaarrfk commented 1 year ago

Green from me. I'm not the biggest expert in QT so I defer to all of you!

Tobias-Fischer commented 1 year ago

Hiya @traversaro - sorry to bother you, you are just too much of a Windows expert. Is there anything suspicious you can see in https://bugreports.qt.io/browse/QTBUG-107009 regarding the location of the .dll / .lib / *.exe files?

traversaro commented 1 year ago

Hiya @traversaro - sorry to bother you, you are just too much of a Windows expert. Is there anything suspicious you can see in https://bugreports.qt.io/browse/QTBUG-107009 regarding the location of the .dll / .lib / *.exe files?

I am relatively new to this, but reading the discussion in https://bugreports.qt.io/browse/QTBUG-107009 probably a workaround that we could use in the conda build script is to add build_dir/$INSTALL_ARCHDATADIR/bin to the PATH before invocking cmake/cmake --build .?

jschueller commented 1 year ago

update: qt-main and qt6-main can now be co-installed even on win32 so we would at least avoid breaking qt5-only packages (and packages wanting to go to qt6 may choose to require qt6-main instead of qt-main)

jschueller commented 1 year ago

There is also the new PySide6 package to think about: https://github.com/conda-forge/staged-recipes/pull/20673 @ccordoba12 seems to think we need some migration path before it can be made available I'm rather in favor in making it available asap (either in a new feedstock or in a branch of the pyside2 recipe like we did for qt6-main) Any thoughts on this ?

Tobias-Fischer commented 1 year ago

I’m in favour of a new branch in the existing feedstock to mirror qt-main and qtwebengine.

hoechenberger commented 1 year ago

I'm rather in favor in making it available asap

I'd love to see that happen :)

jschueller commented 1 year ago

what do you think @hmaarrfk (you also maintain pyside2) ?

hmaarrfk commented 1 year ago

Branch is fine. I don't think the bot is working too well for pyside2

jschueller commented 1 year ago

branch it is then: https://github.com/conda-forge/pyside2-feedstock/pull/136

jschueller commented 1 year ago

qt3d is too big to fit in the 6h constraint (mostly when cross compiling for osx/arm & aarch64) what if we provide another package with the less common packages, let's say qt6-extra with: qt3d and move qtpositioning, qtscxml, qtsensors, qtserialport and qtremoteobject to that new package ? should it have a new branch here also ?

hmaarrfk commented 1 year ago

new package

would the idea be to make it a mandatory dependency? qt6-extra is not very descriptive.

Previously, some expressed that they didn't want to go the "fedora" route, but, I personally like the granularity and how it limits the growth of the qt-main package in the long run.

jschueller commented 1 year ago

no, it would be just a separate package (that depends on qt6-main) like qt-webengine & qt-main qt6-misc ?

hmaarrfk commented 1 year ago

You are very likely going to be the one implementing the work, so i defer to you, but would it be possible to create 5 different packages to follow upstreams naming?

Etc. Seems like a bigger up front setup cost, but it seems like it will be easier to communicate to others, and easier to maintain in the long run.

jschueller commented 1 year ago

ok, let's do that https://github.com/conda-forge/staged-recipes/pull/21050

isuruf commented 1 year ago

Was qt3d in qt-main=5.15?

jschueller commented 1 year ago

yes, at least on linux

isuruf commented 1 year ago

Then splitting packages like this from qt-main is not really a good idea. My suggestion is to break up into individual packages and let qt-main be a meta package like your first qt=6 PR.

jschueller commented 1 year ago

we cannot split completely easily all modules because host tools need to be natively compiled when cross-compiling for at least qtbase+qtdeclarative+qtshadertools+qttools, so we would need at least one package for these 4, this can be circumvented possibly with multiple outputs, but that makes packaging a bit harder also.

I understand your concern that the new qt6-main does not provide qt3d anymore and that may confuse maintainers though, but I also prefer to keep things simple to maintain.

hmaarrfk commented 1 year ago

has anybody noticed that qt3d was missing? Maybe the few visualizations softwares that still aren't migrated.

jschueller commented 1 year ago

I re-thought about it, and imho qt-3d does not seem to be used that much, I only found qgis, maybe you're mistaken with the qopengl classes which are part of qtbase.

hmaarrfk commented 1 year ago

What should we do about things like opencv, a while back, i had something for 5.9, 5.12, and none. Should I revive this effort? https://github.com/conda-forge/opencv-feedstock/pull/206

h-vetinari commented 1 year ago

So the qt 5.15 migration got closed, and so did the migration for qt6 6.5.

Now that we have different & co-installable outputs qt & qt6, I'm not sure how the longterm migration plan looks like? Add a piggyback migrator that rewrites "- qt" in meta.yaml to "- qt6"?

hmaarrfk commented 1 year ago

I don't want to push things in one way or an other yet. I've been testing applications and many have growing pains with QT6. I would like to wait before migrating. OpenCV for one already has enough "double builds".

h-vetinari commented 11 months ago

Coming back ~6 months later, a lot of qt5-things are starting to fall off the wagon. For example, pyside2 isn't really working anymore (missing migrations), whereas pyside6 is fine. This is not a criticism at all, just an indicator of the state of things, so I'm wondering if perhaps the time to migrate to qt6 has come (closer).

hmaarrfk commented 11 months ago

I use pyside2 on a daily basis. not sure what "doesn't work anymore" means.

we are missing qtwebengine for qt6 so we can't just switch over. Spyder uses it.

I could use some help finding a way to build out opencv.

h-vetinari commented 11 months ago

Well, the branch for 5.x on the pyside2-feedstock is not particularly alive (whereas the pyside6 branch gets regular updates). I looked at it because of a user-report that

conda install pyside2 ant python-ldap "openjdk>=17"

was running into conflichts. Might be related to missing migrations for libxml or alsa or whatnot.

hmaarrfk commented 11 months ago

we haven't been updating the version of at (and I don't want to use unsynchronized pyside2 and at versions) but the migrations have been happening.

the problem is that the windows builds of qtwebengine time out, and as such, cannot be built. this will very likely be true for qt6 as well, we just don't build qtwebengine so we haven't run into this yet.

h-vetinari commented 3 months ago

In https://github.com/conda-forge/opencv-feedstock/pull/417, @hmaarrfk asked whether we want to drop qt5 builds (Context: OpenCV has a massive build matrix), leading to the following conversation:

@h-vetinari: Fine by me per se. What's the status on the overall qt6 migration? How many feedstocks (if any) are still qt5-only? I'd kinda like to drop qt5 in general...

@hmaarrfk: Spyder is in general PyQt (not sure if they handle 6) and require webengine. Which we don't have for Qt6.

However. All platforms have a "headless" option. So it should be co-installable, just without the GUI parts on opencv.

I wanted to bring this back into a bit of a wider forum. I'm a bit hesitant to drop qt5 variants if there are important use-cases that cannot yet be supported with qt6 (but at the same time the OpenCV CI situation really asks for a reduction).

I fixed some conflicts in https://github.com/conda-forge/qt-webengine-feedstock/pull/46 for the webengine side, but don't know the status for spyder etc. @ccordoba12?

hmaarrfk commented 3 months ago

See: https://github.com/spyder-ide/spyder/issues/20201

ccordoba12 commented 3 months ago

I fixed some conflicts in https://github.com/conda-forge/qt-webengine-feedstock/pull/46 for the webengine side, but don't know the status for spyder

Spyder should be working with PyQt6 because a volunteer send us several PRs to support it. But we haven't tested it, so I'm not really sure.

The problem for us in Conda-forge is Webengine because we depend on it in several places.

traversaro commented 3 months ago

For pcl we are migrating to qt6 in https://github.com/conda-forge/pcl-feedstock/pull/65 as there is the constraint of using the same qt version used by vtk.

Krakonos commented 2 weeks ago

Hi! When considering different modules, I suggest each of them gets a its own package based on theqt structure. This has a huge advantage for developers, since they already know the package name and don't have to search. The dependency list already is written in the CMakeLists.txt for the project (well, mostly) .

I'm currently missing the NetworkAuth package and had to spend considerable time just to find out it's not available in the repository (I think, I don't have a good way of verifying).

I'd be happy to help out, but TBH I'm not familiar with conda-forge package creation and the process, so I'm kind of lost in the many closed PRs about the qt packages. Is there an active effort to get other packages into conda?