LLVM is required to build PySide2 on Windows

[fbjorn]

I'm trying to build PySide2 from sources on Win10 machine following these steps: Windows conf But when I run
python setup.py bdist_wheel --ignore-git --qmake="C:\Qt\Qt5.9.0\5.9\msvc2015_64\bin\qmake.exe" --cmake="C:\Program Files\CMake\bin\cmake.exe"
I get such an error:

  LLVM version 3.9 is required (llvm-config detected Process failed because:
  for command: llvm-config --version at Process failed because:
  for command: llvm-config --prefix).

Based on your guide, you're not using LLVM. Do you have any thoughts what I'm doing wrong? Thanks

[fredrikaverpil]

I'll have to dig into this (which unfortunately I do not have time with right now). But no, I wasn't using LLVM back when I built PySide2 on Windows last time.

I'd recommend checking in with the PySide2 devs at http://gitter.im/PySide/pyside2?at=5926e5bd00efc2bb3ea3b3ae

[fbjorn]

Just FYI,
I figured out that there are prebuilt clang packages, they provide everything I need.
But anyway there are problems with compilation:

[ 95%] Running generator for 'shiboken2'...
clang_parseTranslationUnit2(0x0, cmd[6]=-fms-compatibility-version=19 -std=c++14 -fPIC -fno-exceptions -Wno-constant-logical-operand C:/Users/dev/AppData/Local/Temp/nothing_Hp2092.hpp)
Errors in C:\Users\dev\AppData\Local\Temp\nothing_Hp2092.hpp:
:0: error: unsupported option '-fPIC' for target 'x86_64-pc-windows-msvc'

Clang: 1 diagnostic messages:
  :0: error: unsupported option '-fPIC' for target 'x86_64-pc-windows-msvc'

Keeping temporary file: C:\Users\dev\AppData\Local\Temp\nothing_Hp2092.hpp
shiboken: Error running ApiExtractor.
Command line: --project-file=C:/Users/dev/pyside-setup/pyside3_build/py3.5-qt5.9.0-64bit-release/shiboken2/shibokenmodule/shibokenmodule.txt
NMAKE : fatal error U1077: C:\Users\dev\pyside-setup\pyside3_build\py3.5-qt5.9.0-64bit-release\shiboken2\generator\shiboken2.exe :   "0x1"
Stop.
NMAKE : fatal error U1077: "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\amd64\nmake.exe" :   "0x2"
Stop.
NMAKE : fatal error U1077: "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\amd64\nmake.exe" :   "0x2"
Stop.
error: Error compiling shiboken2

The circumstances force me to use PyQt5 for now

[fredrikaverpil]

I actually made a little write-up on the alternative that is PyQt5 today: https://fredrikaverpil.github.io/2017/06/30/an-alternative-to-building-pyside2-from-source/

[leycec]

</sigh>

No, the solution is not to install PyQt5 (which is GPL-encumbered, which defeats the whole point of installing PySide2 in the first place) but to install PySide2 correctly: which means install the stable 5.6 branch of PySide2's pyside-setup.git repository rather than that repository's unstable 5.9 or dev branches.

Also, note that the prior pyside.git and shiboken.git repositories have been obsoleted by the pyside-setup.git repository, which merged the contents of the former into itself. (Don't ask, "...but why?" I have utterly no idea either.)

As its label implies, the 5.6 branch requires Qt 5.6.x but otherwise behaves as expected – both during installation and usage. The 5.9 and dev branches, however, require a clang-enabled shiboken2 parser well-known to be dysfunctional – both during installation and usage – for the past several months. Obviously, we hope for this to change... at some point.

tl;dr

Use the pyside-setup.git repository's stable 5.6 branch. Happiness will follow shortly.

[leycec]

@fredrikaverpil Incidentally, would updating the Bintray-hosted wheels to the most recent commit of the 5.6 branch of the pyside-setup.git repository be feasible?

Time scarcity probably remains an issue. As a fellow GitHubber, I grok that. If you simply don't have the time, you simply don't have the time.

But the wheels are nearly a year old at this point. PySide2 has seen dramatic stability improvements and feature additions in the meanwhile. The 5.6 branch is sufficiently stable that I'm surprised (and more than a little disappointed) that The Qt Company hasn't already bundled it as PySide2's official first release.

[fredrikaverpil]

@leycec I somewhat share your frustration. Yes, we could try updating the wheels using the 5.6 branch.

[fredrikaverpil]

So far so good: https://bintray.com/fredrikaverpil/pyside2-wheels/development/5.6#files

[fredrikaverpil]

@fbjorn @leycec Ok, so right now I'm successfully building all the wheels using the 5.6 branch except for the macOS builds. Unfortunately, these builds time out. It's a little strange, as they didn't time out last time around, but maybe PySide2 grew...

I'll try to get the 5.9 branch to build too as it'll require the libclang from Qt, available here. But I'd like to see if I can speed up these macOS builds first.

[leycec]

Well, that escalated quickly. Thanks for being so on-the-ball with this, @fredrikaverpil! You are the Qt superstar this world needs.

We've had continued luck with the 5.6 branch on Gentoo Linux as well. The 5.9 and dev branches? Not so much.

If you are able to eventually get either of the latter two branches up and running, that would be an extremely useful finding. We'd probably immediately follow suite on Gentoo. Until then, 5.6 on macOS sounds like a grand idea. Since neither Homebrew nor MacPorts currently provide a PySide2 package, the wheels produced by this esteemed project are pretty much the only sane means of installing PySide2 on macOS.

[fredrikaverpil]

Well, that escalated quickly.

Expect this to get merged into the develop branch in the next couple of hours. The wheels are available in the development bintray. After having merged with master, wheels will get built into the main bintray. I'll also update the quickstart info in the README when those wheels have been built.

You are the Qt superstar this world needs.

Hehe, thanks! 💪

The 5.9 and dev branches? Not so much.

Did you use the QtC-prebuilt libclang when attempting this?
Any chance you could share your build setup?

[leycec]

Did you use the QtC-prebuilt libclang when attempting this?

Nope! That's a sane idea, though; I wasn't even aware that QtC had released libclang binaries until stumbling across your reference above.

Do you know exactly what those binaries provide? Are they merely the official versions of libclang pre-compiled for various popular platforms supported by Qt or have they been internally patched in some way to meet Qt-specific requirements? My Google-fu failed me here.

I suspect they're merely the official versions of libclang – which, while still useful, wouldn't be quite as useful for us. Gentoo Portage already packages all versions of libclang officially supported by Qt, including 3.8.1, 3.9.1, and 4.0.0. Also, because we're all intractable fundamentalist zealots here in Gentoo Linux Land, ^{...won't you join us?} pre-built binaries are a point of contention for us. Our entire system is compiled from source by end users at installation time with only a few exceptions for for-profit proprietary software that no one can do without. By which I mean Steam.

Any chance you could share your build setup?

Sure! Since we never succeeded in compiling the 5.9 or dev branches (despite issuing multiple bug requests against the upstream PySide2 tracker), I can only admit that our setup is probably obsolete and unlikely to help anyone. Rage and anger would quickly follow.

Our key finding was that the top-level sources/shiboken2/CMakeLists.txt file failed to properly detect libclang installations and had to be explicitly instructed where to find libclang – namely, by exporting the CLANG_INSTALL_DIR environment variable on invoking cmake: e.g.,

CLANG_INSTALL_DIR=/usr/lib64/clang/3.9.1/ cmake

Since upstream has hopefully since resolved this (e.g., by actually calling find_package(CLang) like they really should have done in the first place), I'd try banging on cmake without exporting that variable first.

Sadly, PySide2's July 20th update suggests that libclang issues still linger:

- 5.9 still having clang header issues

Why. Why? WHY!?! ^{...ok, i feel better now}

[fredrikaverpil]

Do you know exactly what those binaries provide?

Nope, sorry. But if you haven't used those, I wouldn't expect anything to work.

Our entire system is compiled from source by end users at installation time with only a few exceptions for for-profit proprietary software

Wowsers. I actually didn't know that. Is a Gentoo install compiling everything from source?!
Sounds like it'll take ages to install?

Sadly, PySide2's July 20th update suggests that libclang issues still linger

Hm. I checked today and the devs seem to be able to build PySide2 5.9 just fine all the time... I haven't looked into it yet so I don't know... but perhaps I should give it a week or two before attempting it then.

[leycec]

But if you haven't used those, I wouldn't expect anything to work.

</sigh>

Is a Gentoo install compiling everything from source?!

Religiously so. We're probably the best known source-based distribution, with a number of downstream distributions nobody has ever heard of: Calculate Linux, CoreOS, Sabayon, and (of course) Google's Chromium OS.

Other source-based *nix platforms only spiritually related to Gentoo include Linux from Scratch, all of the BSD variants (e.g., FreeBSD, NetBSD, OpenBSD), and all of the macOS package managers (e.g., Homebrew, MacPorts). Other Linux package managers like Arch's pacman also provide optional support for source-based installations (e.g., via the Arch User Repository (AUR)), which is nice.

While purely binary-based package managers still predominate, the landscape is gradually diversifying. Source-based package managers offer a number of significant advantages (like hardware optimization, security hardening, and extreme ricing) and an equal number of significant disadvantages (mostly, installation space and time).

Sounds like it'll take ages to install?

It does, at least on traditional HDDs. This is mitigated by migrating to some combination of SDDs (easy but costly) and/or ramdisks (hard but cheap, assuming sufficient RAM). Compiling to a ramdisk is sufficiently fast for the typical package. ^{...which means anything not Chromium, Firefox, or webkit.}

Gentoo Portage is no apt-get and never will be, but it doesn't necessarily need to be. It only needs to be fast enough – which, thanks to utopian technological progress, it increasingly is. Sort of.

But... yes. The high cost of installation-by-compilation remains our Achilles heel. It's a hopeless uphill battle, but we tend to feel that the subtle benefits of a Poettering-free distribution outweigh the obvious drawbacks.

But we would say that. We're fanatics! ⛪️

[leycec]

I checked today and the devs seem to be able to build PySide2 5.9 just fine all the time...

O.K.! That's great to hear, actually. Thanks for the positive confirmation. Since our last attempts to support PySide2 5.9 failed in an apocalyptic inferno of developer flames and upstream rants, we'll probably hold off until mid- to late-August before trying again.

But I fully support you trying. If anybody can make this work, it'll be pyside2-wheels. ☸️

[fredrikaverpil]

Religiously so. We're probably the best known source-based distribution

Wow, I had no idea. Sounds both awesome and frightening at the same time. For a project such as this, I'm pretty sure it wouldn't make much sense to build PySide2 using a pre-built Qt5 on Gentoo then?

we'll probably hold off until mid- to late-August before trying again.

I might do that too.

[fredrikaverpil]

I'm going to close this issue, as it was originally about not being able to compile on Windows without LLVM and that the solution to that is (most likely) that you need to git checkout 5.9 and use the QtC-provided libclang binaries for 5.9: http://download.qt.io/development_releases/prebuilt/libclang ...or you can skip it but build the 5.6 branch instead.

I'm tracking building the 5.9 branch in a separate issue: https://github.com/fredrikaverpil/pyside2-wheels/issues/63

Feel free to subscribe to that issue for updates.