32-bit QML Application not working on Windows 7

[Siecje]

I created an installer using the code from https://github.com/siecje/qml-testing/tree/pynsist with the bitness=32 in installer.cfg

I installed it on the IE8 on Win7 machine from here

https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/

When I launch the shortcut nothing happens. If I run it from Python

> Python\pythonw.exe QMLApplication.launch.pyw

In %appdata%\QMLApplication.launch.pyw

there is

Traceback (most recent call last):
  File "QMLApplication.launch.pyw", line 30, in <module>
    from qmlapp import Main
  File "pkgs\qmlapp\__init__.py", line 1, in <module>
    from .main import Main
  File "pkgs\qmlapp\main.py", line 5, in <module>
    from PyQt5 import QtCore, QtGui, QtQml
ImportError: DLL load failed: The specified module could not be found.

[takluyver]

(Aside: in the command line, you can run with python.exe instead of pythonw.exe to get tracebacks directly in the command prompt)

Can you try:

1. Split that import statement out into separate lines so we can see which module is actually causing the problem.
2. Use dependency walker on the .pyd file having the problem, and see if you can work out what DLL is missing.

[Siecje]

Okay I was wondering why I wasn't getting output in the console.

All of the imports fail with the same message. It just depends which one it tries first.

I opened C:\Program Files\QMLApplication\Pkgs\PyQt5\QtCore.pyd in Dependency Walker.

It said it was missing some .dll files, but some of them were in the Python directory. I added C:\Program Files\QMLApplication\Python\ to the search path.

It says it is missing

Qt5Core.dll
IEShims.dll

Note: IEShims.dll was not present before adding the Python directory to the search path.

[takluyver]

IEShims.dll can apparently be safely ignored.

Qt5Core.dll is there under PyQt5/Qt/bin/Qt5Core.dll. The __init__.py modifies PATH to add that directory so that it can find those DLLs. Maybe something's going wrong with that PATH modification mechanism - but you already had this working with 64-bit Python, right? Both wheels use the same mechanism.

Maybe it's something further along. Does loading Qt5Core.dll in dependency walker show anything interesting?

[Siecje]

When I open PyQt5/Qt/bin/Qt5Core.dll and add C:\Program Files\QMLApplication\Python as a search path the only missing modules are

CONCRT140.DLL
IESHIMS.DLL

Also in the Log Window I get

Warning: At least one delay-load dependency module was not found.
Warning: At least one module has an unresolved import due to a missing export function in a delay-load dependent module.

[takluyver]

CONCRT140.dll sounds like yet another part of the MSVC runtime, which is gradually becoming Pynsist's nemesis. The PyQt5 wheels include msvcp140.dll, which I believe is the C++ part of the runtime, but neither PyQt5 nor the Python embeddable build appears to include CONCRT140.

@zooba, sorry to bother you again, but what is CONCRT, is it likely to be required for a PyQt5 application, and where should it be coming from?

The log messages don't mean much to me, but I don't know this stuff well.

[Siecje]

@takluyver Thanks a lot for your help!

For the record I have not tested the 64bit installer on a fresh 64bit Windows installation. I found the IE test VMs and the first one I downloaded for IE 8 was 32bit and decided to test there.

I found this https://msdn.microsoft.com/en-us/library/ee207192.aspx#runtime

I'm not sure if it is related but it says some of the functionality has been replaced by Windows ThreadPool and the application does use the Python threading module.

[takluyver]

Aha, CONCRT = concurrency runtime. I doubt it's related to Python's threading support, but I wouldn't be surprised if Qt uses it.

In your Windows installation where this is failing, can you install Python 3.5, pip install PyQt5 into it, and see if you can import PyQt5.QtCore there? If that fails as well, then it may be something that Riverbank is missing when making the wheels.

[Siecje]

Oddly I tried to install Python 3.5.2 32bit and it got stuck on

installing C Runtime Update (KB2999226)

It was stuck for about an hour and svchost.exe had 100% CPU and 800 MB RAM.

The install log contained

#...
[0C8C:0EE4][2016-11-18T10:43:14]i305: Verified acquired payload: path_JustForMe at path: C:\Users\IEUser\AppData\Local\Package Cache\.unverified\path_JustForMe, moving to: C:\Users\IEUser\AppData\Local\Package Cache\{7E08C4EE-B1C7-4138-8227-7CD3837636AA}v3.5.2150.0\path.msi.
[0D24:0C28][2016-11-18T11:21:02]e000: Error 0x80070642: Bootstrapper application aborted during MSU progress.
[0D24:0C28][2016-11-18T11:21:02]e000: Error 0x80070642: Failed to execute MSU package.
[0C8C:0668][2016-11-18T11:21:02]e000: Error 0x80070642: Failed to configure per-machine MSU package.
[0C8C:0668][2016-11-18T11:21:02]i319: Applied execute package: crt_14.0_v6.1_x86, result: 0x80070642, restart: None
[0C8C:0668][2016-11-18T11:21:02]e000: Error 0x80070642: Failed to execute MSU package.
[0D24:0C28][2016-11-18T11:21:02]i351: Removing cached package: crt_14.0_v6.1_x86, from path: C:\ProgramData\Package Cache\D4036846864773E3D647F421DFE7F6CA536E307B\
[0C8C:0668][2016-11-18T11:21:02]i372: Session end, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{cf72a2ab-2f1d-49fd-a0d7-1065e6357e1e}, resume: None, restart: None, disable resume: No
[0C8C:0668][2016-11-18T11:21:02]i330: Removed bundle dependency provider: {cf72a2ab-2f1d-49fd-a0d7-1065e6357e1e}
[0C8C:0668][2016-11-18T11:21:02]i352: Removing cached bundle: {cf72a2ab-2f1d-49fd-a0d7-1065e6357e1e}, from path: C:\Users\IEUser\AppData\Local\Package Cache\{cf72a2ab-2f1d-49fd-a0d7-1065e6357e1e}\
[0C8C:0668][2016-11-18T11:21:02]i371: Updating session, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{cf72a2ab-2f1d-49fd-a0d7-1065e6357e1e}, resume: None, restart initiated: No, disable resume: No
[0C8C:0668][2016-11-18T11:21:02]i399: Apply complete, result: 0x80070642, restart: None, ba requested restart:  No

Then I reverted the VM to the same snapshot and tried again and it installed without any problems.

When I run python I get an error window that says

The program can't start because api-ms-win-crt-runtime-L1-1-0.dll is missing from your computer. Try reinstalling the program to fix this problem.

I have tried reverting to the first snapshot (fresh download of the VM) and I get the same thing.

[Siecje]

Okay so apparently the Python 3.5.x installers only work out of the box on Window 10+

I had to install the Microsoft Visual C++ 2015 Redistributable (x86) - 14.0.23026

in order to just run python

After that I was able to python -m ensurepip and pip3 install PyQt5

I was able to import

from PyQt5 import QtCore
from PyQt5 import QtGui
from PyQt5 import QtQml

[Siecje]

After installing Microsoft Visual C++ 2015 Redistributable (x86) - 14.0.23026

CONCRT140.dll is located in C:\Windows\System32\

And the Application works!

So does Pynsist need to include the CONCRT140.dll from the Visual C++ 2015 Redistributable?

Downloads links https://www.microsoft.com/en-ca/download/details.aspx?id=48145

Update: Nevermind it was Qt5Core.dll that required it. So Pynsist doesn't need to do anything different.

[zooba]

The Python 3.5 installers will install the required update on pre-Windows 10 machines. Windows 10 already includes the necessary files, so it doesn't need to do it.

CONCRT140.dll is part of the C++ runtime, which is not part of the C Runtime update that gets installed. Python does not depend on it, so PyQt5 should either include it or specify that you need to install the C++ redist to get it. Alternatively, whoever builds it could statically compile these DLLs into the PyQt5 binary.

That said, if pynsist starts installing the C++ runtime everywhere, installs will be bigger, take longer, require admin privileges, and be more likely to fail, but once you get past all that then you'll have both C and C++ runtimes and it won't matter what dependencies need. (Up until those dependencies switch to a new compiler version, and suddenly you need CONCRT141.dll or something - this is why I decided that Python will only include its own dependencies and packages should include their own or otherwise require them.)

[Siecje]

The Python 3.5 installers will install the required update on pre-Windows 10 machines.

@zooba Any idea why I wasn't able to run python without installing the Visual C++ Runtime?

It was in my PATH and the error message even happened when I double clicked python.exe

The error was about api-ms-win-crt-runtime-L1-1-0.dll missing. Which comes from the C++ runtime.

[zooba]

@Siecje If you successfully installed it using the installer, then I'd need to see all your logs (%TEMP% directory).

If you used the embeddable ZIP file then you need to install the C runtime yourself - installing the C++ runtime is one way to get it. Installing all available Windows Updates is another way, as it was made available all the way back to Windows Vista via WU.

[Siecje]

@zooba I emailed you the logs with a full explanation of the steps.

Thanks a lot @zooba and @takluyver for helping me figure out what was going on. Thanks so much!!

[takluyver]

Thanks Steve,

CONCRT140.dll is part of the C++ runtime, which is not part of the C Runtime update that gets installed. Python does not depend on it, so PyQt5 should either include it or specify that you need to install the C++ redist to get it.

PyQt5 already includes msvcp140, which I believe is part of the C++ runtime, in the Windows wheels. If it's already including one C++ runtime DLL, I'd guess it makes sense for it to include concrt140 as well? It might reach out to the PyQt developers about this.

To check my understanding: the Visual C++ redistributable includes the C and C++ runtimes, right? I've been pointing people to this in the docs if they don't want to bundle the universal C runtime DLLs into their installer.

[zooba]

To check my understanding: the Visual C++ redistributable includes the C and C++ runtimes, right?

That's right. The C runtime is a prerequisite for the C++ runtime, so it makes sure you have both. (Though if you're a Python 3.5+ extension, you can assume that the C runtime is already available.)

[takluyver]

Thanks @Siecje - I was just going to ask about including CONCRT140 in the PyQt5 wheels, but I see you've already done it. :-)

[Siecje]

There is a new version of PyQt5, version 5.7.1 which includes CONCRT140.dll in the wheel.

I've created a new 32-bit installer using this code

https://github.com/Siecje/qml-testing/tree/e2b71e13396c0a8cc9eafbc06fc502b953534e60

When I run the shortcut in the start menu, nothing happens, no error window or anything.

%appdata%\QMLApplication.launch.pyw contains

Traceback (most recent call last):
  File "C:\Program Files\QMLApplication\QMLApplication.launch.pyw", line 30, in <module>
    from qmlapp import Main
  File "C:\Program Files\QMLApplication\pkgs\qmlapp\__init__.py", line 1, in <module>
    from .main import Main
  File "C:\Program Files\QMLApplication\pkgs\qmlapp\main.py", line 5, in <module>
    from PyQt5 import QtCore
ImportError: DLL load failed: The specified module could not be found.

I have looked at the QtCore.pyd and Qt5Core.dll files in Dependency Walker but there doesn't seem to be anything missing.

Two of the dependencies have a red icon.

[takluyver]

That's puzzling. Is there any information in the UI on what the red icons mean? Does it work if you install Python and pip install PyQt5 again?

[Siecje]

I can't figure out why those two files have a different icon.

When I install Python 3.6.0 or Python 3.5.2 on this VM I get an error about missing api-ms-win-crt-runtime-L1-1-0.dll

This is what I am doing:

1) I downloaded the VMWare VM for IE 8 on Win 7 from https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/ 2) Activated Windows for 90 days by running slmgr /ato in a administrative command prompt (Just incase an invalid license would prevent a request for a Windows update to be installed) 3) Downloaded Python 3.6.0 installer https://www.python.org/ftp/python/3.6.0/python-3.6.0.exe 4) Run installer and checked add Python to PATH during install wizard Install location is C:\Users\IEUser\AppData\Local\Programs\Python\Python36-32 by default. 5) Open cmd.exe and run python Get error window in the attached image. 6) Restart Windows 7) Open cmd.exe and run python Get error window in the attached image.

I have attached the Python logs found in %temp%

Python 3.6.0 (32-bit)_20170106053853_logs.zip

[Siecje]

The same error happens when I download the embeddable zip file and extract it and run python.exe

https://www.python.org/ftp/python/3.6.0/python-3.6.0-embed-win32.zip

[takluyver]

@zooba : the description @Siecje gives above sounds like the Python installers are failing to install the necessary bits of the CRT for Python itself. Is this something you're aware of already?

I don't think that should affect the Pynsist-built installer, though. That DLL should be packaged and installed in the Python subdirectory of the installation, since you're using the bundled format.

Apologies if I already asked about this, but is this specific to 32-bit? Does it work if you build an installer with 64-bit Python?

[Siecje]

I don't have a bare bones 64-bit Windows 7 to test.

But I have discovered the problem.

Python requires this Windows update https://support.microsoft.com/en-us/kb/2999226

I tried running Windows Update but it was talking a very long time. So I tried installing the update directly but it too was taking a long time, stuck at "Searching for updates on this computer..."

By following the instructions here http://www.freenode-windows.org/resources/vista-7/windows-update

I was able to stop Windows Update and install that update directly. After that Python and the pynsist applications worked!!!

[takluyver]

That fits in with my understanding of why the Python installer doesn't work - it tries to install that update on your system, and evidently it's not working properly. But the Pynsist application shouldn't need that, as far as I know - it should bundle those libraries.

What version of Pynsist were you building with? if it's older than 1.9, can you upgrade and try again? I changed how the CRT is handled in 1.9.

[Siecje]

That was with version 1.9

The problem was that even the embeddable Python was not working.

Are you saying that pynsist will install the files that are found in this Windows Update? https://support.microsoft.com/en-us/kb/2999226

[takluyver]

Yep, it should install them if it doesn't find ucrtbase.dll already present on the system. Here they are in the package:

https://github.com/takluyver/pynsist/tree/master/nsist/msvcrt/x86

Can you look into the Python subdirectory under the install directory, and see whether they are installed?

[takluyver]

To clarify, I'd ideally like you to look in the install directory on a system where it's not working.

[Siecje]

Yes ucrtbase.dll is there.

Edit: Yes the .dll files are there, including the one that the installed Python is complaining about.

I am able to run the python.exe in the install directory.

This is before installing the Windows Update.

[takluyver]

So you can run that python.exe but it can't import PyQt5 without the update installed? That sounds to me like PyQt5 is still missing some DLL that it requires in that situation, like it was missing CONCRT before. I don't know how to find out what that is, though.

[Siecje]

What else does Window Update KB2999226 do besides installing the files the installer is adding?

https://github.com/takluyver/pynsist/tree/310af49bb98a61ab0a9c1cd52a137453f559fcd5/nsist/msvcrt/x86

If I install the update the pynsist installed application works. So it seems like the PyQt5==5.7.1 wheel is missing something in that update.

[Siecje]

According to the list of files for KB2999226 on Windows 7 x86 here

https://support.microsoft.com/en-ca/kb/2999226

Under "For all supported x86-based versions of Windows 7"

The folder in the pynsist repo
https://github.com/takluyver/pynsist/tree/310af49bb98a61ab0a9c1cd52a137453f559fcd5/nsist/msvcrt/x86

is missing

api-ms-win-core-xstate-l2-1-0.dll
api-ms-win-eventing-provider-l1-1-0.dll

From fresh Windows 7 I install the pynsist application and then copy those two files into the Python sub directory of the install directory and it doesn't work.

Nothing happens when I launch the shortcut but the file in %appdata%

contains

Traceback (most recent call last):
  File "C:\Program Files\QMLApplication2\QMLApplication.launch.pyw", line 30, in <module>
    from qmlapp import Main
  File "C:\Program Files\QMLApplication2\pkgs\qmlapp\__init__.py", line 1, in <module>
    from .main import Main
  File "C:\Program Files\QMLApplication2\pkgs\qmlapp\main.py", line 5, in <module>
    from PyQt5 import QtCore
ImportError: DLL load failed: The specified module could not be found.

[Siecje]

Maybe there are more files that are not listed on that page?

[takluyver]

Hmm, that's annoying. I probably got the files on my Windows 8.1 installation. I would have thought the SDK would include all the necessary ones, though.

But since copying them doesn't change things, it doesn't sound like they're the issue here.

[zooba]

I am aware of some issues where the update reports that it is installed, but it actually requires a reboot to finish installing. On a freshly created machine this is fairly likely, as it only really occurs when other updates have been installed and you haven't rebooted yet. The embedded distro is just as affected by this, as it requires you to manage the installation yourself rather than having the installer do it.

The only files included in that update should be ucrtbase.dll and api-ms-win-crt-*.dll files (vcruntime140.dll is included with Python, even the embeddable package). If they are api-ms-win-(not crt)-*.dll then they are system API sets that should be installed by the OS.

What may be happening here is the build is being done targeting a later version of the OS, and so when the PYD comes back to Win7 it is looking for files that are not guaranteed to be there. I'll need to look deeper into how this works, since (from memory) I'm not sure where those options would be, but it may just be the case that those also need to be redistributed with the build. If that's the case, they should be somewhere in a Redist directory on the machine that built it, and I'll have to look at making setuptools/distutils/wheel include them when necessary.

[Siecje]

Is it possible to see which files the update is adding?

The webpage lists Api-ms-win-core-xstate-l2-1-0.dll twice so maybe there is another file that gets installed?

[zooba]

Ah, I see. There are some extra files for Win8 and earlier in the KB that are not in the pynsist tree and not in the redist folder in the WinSDK. I'll ping the team that owns this so they can get the files added, but otherwise I'd suggest finding them on a Windows 7 or 8 (not 8.1) machine with KB2999226 installed and copying them from there.

[Siecje]

How do you find them?

[zooba]

They'll be in C:\Windows\System32 for the 64-bit version and C:\Windows\SysWOW64 for the 32-bit version (yes, I know the names are backwards).

I assume on Windows 8.1 and later the schema is embedded into the loader, so you'll only find the files on Windows 7 or 8 after installing that KB.

[takluyver]

Thanks @zooba. Can you ping me as and when the extra files are in the SDK, and I'll add them to Pynsist?

It sounds like the PyQt issue is something that needs to be fixed in the PyQt wheels. I don't know how to find out what file is missing.

[Siecje]

Is it possible to tell which files came from KB1999116?

After installing and looking in C:\Windows\System32\ the only extra files that start with api-ms- are

api-ms-win-core-xstate-l2-1-0.dll
api-ms-win-eventing-provider-l1-1-0.dll

[zooba]

@Siecje The list of files is in the KB article which you posted earlier.

[Siecje]

Any idea why the application doesn't work until the KB is installed?

Even with all of the .dll files in a directory on the search path.

Does the update do anything else?

[zooba]

The issue was that not all the files were installed - the two that you listed above are missing from Pynsist because they are missing from the Windows SDK.

The SDK team is going to fix their problem. The KB does not do anything other than copy the files into the right place.

[Siecje]

Those two files were installed into C:\Windows\System32\ I copied them into the same directory that pynsist is putting them.

From a bare bones Windows 7 32-bit I installed the pynsist installer (which contains all but two of the api-ms-*.dll files) and then copied the two that were missing into the same location pynsist puts them.

And the application still doesn't work. But if I install the KB then it works.

I'm trying to figure out why that is.

[takluyver]

Yeah, I don't understand how that collection of facts fits together... 😕

[zooba]

@Siecje Did you take the files from C:\Windows\System32 on 64-bit Windows?

[Siecje]

No I took it from the same 32-bit Windows that it was working on.

I have a VM with snapshots and I installed the KB then copied the files out of the VM and then reverted to a snapshot before installing the KB and placed the files in the Python directory in the pynsist directory with the other .dll files.

[Siecje]

If I move the files from the Python directory to C:\Windows\System32\ then it works.

Steps

1) Install pynsist installer 2) Application does not work 3) Copy api-ms-*.dll files from C:\Program Files\QMLApplication2\Python to C:\Windows\System32 4) Application now works.

This is without installing the KB.

[zooba]

Hmm... perhaps they need to be able to locate some other files in the system directory?

What is the error message you get with those files in the application directory? Or better yet, what does depends.exe say is missing?

[Siecje]

No error window or anything. depends.exe does not show anything missing.

In Python the error is about

from PyQt5 import QtCore
ImportError: DLL load failed: The specified module could not be found.