pyinstaller / pyinstaller-hooks-contrib

Community maintained hooks for PyInstaller.
Other
96 stars 126 forks source link

Issue with PyInstaller: cryptography.hazmat.bindings._rust module not found during packaging #809

Closed Marlon2025c closed 2 months ago

Marlon2025c commented 2 months ago

Description of the issue

I am encountering an issue when using PyInstaller with the cryptography library. Everything works fine when running the script in .py format, but once I package it as an executable, I encounter the following error:

Traceback (most recent call last):
  File "spaziacraft.py", line 9, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 378, in exec_module
  File "cryptography.fernet.py", line 14, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 378, in exec_module
  File "cryptography.exceptions.py", line 9, in <module>
ModuleNotFoundError: No module named 'cryptography.hazmat.bindings._rust'

I have tried using --hidden-import=cryptography.hazmat.bindings._rust, but the issue persists.

Context information (for bug reports)

rokm commented 2 months ago

Seems to work as expected in my test environment.

If you try to freeze a basic

import cryptography.fernet
print(cryptography.fernet)

program, does it throw the same error? If it does, can you

a) copy the whole build log here

b) compress the corresponding build and dist directories and upload the archive somewhere so I can take a look at it

Marlon2025c commented 2 months ago

When I import cryptography.fernet and execute the following command:

import cryptography.fernet
print(cryptography.fernet)

I do not encounter any issues. It works as expected in my environment.

rokm commented 2 months ago

Hmm, what's the minimal reproducer, then? What happens on line 9 in spaziacraft.py?

rokm commented 2 months ago

Actually, wait, did you try just running that snippet in your environment, or also freeze it using PyInstaller as well and run the executable?

Marlon2025c commented 2 months ago

Inspaziacraft.py, at line 9, I have the following line:

from cryptography.fernet import Fernet I used PyInstaller to create the executable and then I ran the .exe file.

rokm commented 2 months ago

Can you rebuild your application with added --clean option and paste the full build log here, then? Along with the full PyInstaller command you are using.

Marlon2025c commented 2 months ago

I executed the following command to build the executable for spaziacraft.py, using the --clean option to remove temporary files:

pyinstaller --clean --onefile spaziacraft.py

Here are the generated logs:

PS C:\Users\basti\OneDrive\Bureau\projet_recupe> pyinstaller --clean --onefile spaziacraft.py
194 INFO: PyInstaller: 6.10.0, contrib hooks: 2024.8
194 INFO: Python: 3.12.6
217 INFO: Platform: Windows-11-10.0.22631-SP0
218 INFO: Python environment: C:\Users\basti\AppData\Local\Programs\Python\Python312
218 INFO: wrote C:\Users\basti\OneDrive\Bureau\projet_recupe\spaziacraft.spec
220 INFO: Removing temporary files and cleaning cache in C:\Users\basti\AppData\Local\pyinstaller
227 INFO: Module search paths (PYTHONPATH):
['C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Scripts\\pyinstaller.exe',
 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\python312.zip',
 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\DLLs',
 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib',
 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312',
 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages',
 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\setuptools\\_vendor',
 'C:\\Users\\basti\\OneDrive\\Bureau\\projet_recupe']
439 INFO: checking Analysis
439 INFO: Building Analysis because Analysis-00.toc is non existent
439 INFO: Running Analysis Analysis-00.toc
439 INFO: Target bytecode optimization level: 0
439 INFO: Initializing module dependency graph...
442 INFO: Caching module graph hooks...
452 INFO: Analyzing base_library.zip ...
1725 INFO: Processing standard module hook 'hook-encodings.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
2949 INFO: Processing standard module hook 'hook-pickle.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
3757 INFO: Processing standard module hook 'hook-heapq.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
4037 INFO: Caching module dependency graph...
4111 INFO: Looking for Python shared library...
4134 INFO: Using Python shared library: C:\Users\basti\AppData\Local\Programs\Python\Python312\python312.dll
4134 INFO: Analyzing C:\Users\basti\OneDrive\Bureau\projet_recupe\spaziacraft.py
4139 INFO: Processing standard module hook 'hook-minecraft_launcher_lib.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\_pyinstaller_hooks_contrib\\stdhooks'   
4480 INFO: Processing pre-safe-import-module hook 'hook-typing_extensions.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module'
4482 INFO: SetuptoolsInfo: initializing cached setuptools info...
8235 INFO: Processing standard module hook 'hook-charset_normalizer.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\_pyinstaller_hooks_contrib\\stdhooks'
8322 INFO: Processing standard module hook 'hook-cryptography.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\_pyinstaller_hooks_contrib\\stdhooks'
9355 INFO: Processing standard module hook 'hook-certifi.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\_pyinstaller_hooks_contrib\\stdhooks'
9528 INFO: Processing standard module hook 'hook-platform.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
9683 INFO: Processing standard module hook 'hook-PyQt6.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
9821 INFO: Processing standard module hook 'hook-PyQt6.QtWidgets.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
10064 INFO: Processing standard module hook 'hook-PyQt6.QtWebEngineWidgets.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
10229 INFO: Processing standard module hook 'hook-PyQt6.QtWebEngineCore.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
10615 INFO: Processing standard module hook 'hook-PyQt6.QtCore.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
10826 INFO: Processing standard module hook 'hook-PyQt6.QtGui.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
11575 INFO: Processing standard module hook 'hook-psutil.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\_pyinstaller_hooks_contrib\\stdhooks'
11725 INFO: Processing module hooks (post-graph stage)...
11773 INFO: Processing standard module hook 'hook-PyQt6.QtNetwork.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
12180 INFO: Processing standard module hook 'hook-PyQt6.QtPositioning.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
12409 INFO: Processing standard module hook 'hook-PyQt6.QtPrintSupport.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
12559 INFO: Processing standard module hook 'hook-PyQt6.QtQuick.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
12748 INFO: Processing standard module hook 'hook-PyQt6.QtQuickWidgets.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
12888 INFO: Processing standard module hook 'hook-PyQt6.QtWebChannel.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
13016 INFO: Processing standard module hook 'hook-PyQt6.QtOpenGL.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
13185 INFO: Processing standard module hook 'hook-PyQt6.QtQml.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
15655 INFO: Performing binary vs. data reclassification (1483 entries)
27163 INFO: Looking for ctypes DLLs
27171 INFO: Analyzing run-time hooks ...
27173 INFO: Including run-time hook 'pyi_rth_inspect.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks\\rthooks'
27175 INFO: Including run-time hook 'pyi_rth_cryptography_openssl.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\_pyinstaller_hooks_contrib\\rthooks'
27176 INFO: Including run-time hook 'pyi_rth_pyqt6.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks\\rthooks'
27185 INFO: Processing pre-find-module-path hook 'hook-_pyi_rth_utils.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks\\pre_find_module_path' 
27192 INFO: Processing standard module hook 'hook-_pyi_rth_utils.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks'
27205 INFO: Including run-time hook 'pyi_rth_pkgutil.py' from 'C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyInstaller\\hooks\\rthooks'
27248 INFO: Looking for dynamic libraries
27817 INFO: Extra DLL search directories (AddDllDirectory): ['C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyQt6\\Qt6\\bin']
27817 INFO: Extra DLL search directories (PATH): ['C:\\Users\\basti\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\PyQt6\\Qt6\\bin']
34888 INFO: Warnings written to C:\Users\basti\OneDrive\Bureau\projet_recupe\build\spaziacraft\warn-spaziacraft.txt
34912 INFO: Graph cross-reference written to C:\Users\basti\OneDrive\Bureau\projet_recupe\build\spaziacraft\xref-spaziacraft.html
34958 INFO: checking PYZ
34958 INFO: Building PYZ because PYZ-00.toc is non existent
34958 INFO: Building PYZ (ZlibArchive) C:\Users\basti\OneDrive\Bureau\projet_recupe\build\spaziacraft\PYZ-00.pyz
35320 INFO: Building PYZ (ZlibArchive) C:\Users\basti\OneDrive\Bureau\projet_recupe\build\spaziacraft\PYZ-00.pyz completed successfully.
35370 INFO: checking PKG
35371 INFO: Building PKG because PKG-00.toc is non existent
35371 INFO: Building PKG (CArchive) spaziacraft.pkg
64987 INFO: Building PKG (CArchive) spaziacraft.pkg completed successfully.
65012 INFO: Bootloader C:\Users\basti\AppData\Local\Programs\Python\Python312\Lib\site-packages\PyInstaller\bootloader\Windows-64bit-intel\run.exe
65012 INFO: checking EXE
65012 INFO: Building EXE because EXE-00.toc is non existent
65012 INFO: Building EXE from EXE-00.toc
65013 INFO: Copying bootloader EXE to C:\Users\basti\OneDrive\Bureau\projet_recupe\dist\spaziacraft.exe
65043 INFO: Copying icon to EXE
65070 INFO: Copying 0 resources to EXE
65070 INFO: Embedding manifest in EXE
65097 INFO: Appending PKG archive to EXE
65197 INFO: Fixing EXE headers
65716 INFO: Building EXE from EXE-00.toc completed successfully.
rokm commented 2 months ago

Hmm, nothing unusual in the log...

If you build in onedir mode instead of onefile, does cryptography\hazmat\bindings\_rust.pyd exist in the _internal directory (presumably it does not? Does the cryptography directory exist at all?).

If you open build\spaziacraft\warn-spaziacraft.txt or build\spaziacraft\xref-spaziacraft.html, and search for cryptography.hazmat.bindings._rust, is it listed as a missing module?

Marlon2025c commented 2 months ago

I’ve checked, and the cryptography\hazmat\bindings\_rust.pyd file is present. Capture d'écran 2024-09-24 074547

After reviewing the warn-spaziacraft.txt and xref-spaziacraft.html files, there doesn’t appear to be any missing modules related to cryptography.hazmat.bindings._rust warn-spaziacraft.txt

bwoodsend commented 2 months ago

The only thing I can think of is that one of your other dependencies is pulling in a DLL that is shadowing another DLL that cryptography depends on. I guess just comment out imports until you find the one that breaks it?

rokm commented 2 months ago

Can you share the problematic build of spaziacraft in onedir mode (both build and dist directories) so I can take a closer look at what is going on?

If not, can you come up with a minimal reproducer that you can share? I suppose removing all code from your modules except for imports at the top would be a good starting point... (and at the same time, you can try Brénainn's idea to remove some of non-cryptography imports and see if that fixes cryptopgraphy - which would indeed indicate that there is some sort of mix up going on).

Marlon2025c commented 2 months ago

Hello,

I apologize for the delay in my response. Here’s the debug code I’m using, and it seems that the issue comes from os.execv after converting to .exe (onefile mode). In the .py version, everything works correctly:

import sys, os, configparser
import time
from cryptography.fernet import Fernet

def reload_app():
    print("Reloading app...")
    sys.stderr.write("reload_app\n")
    time.sleep(2)
    os.execv(sys.executable, ['python'] + sys.argv)

def main():
    sys.stderr.write("50\n")
    print("50")
    reload_app()

if __name__ == "__main__":
    main()

The issue arises during the application restart, where temporary files disappear after the call to os.execv. Thank you for your assistance.

rokm commented 2 months ago

Aha, that finally makes sense.

Starting with PyInstaller 6.10, you need to set PYINSTALLER_RESET_ENVIRONMENT environment variable if you want to spawn an independent instance of application: https://pyinstaller.org/en/stable/common-issues-and-pitfalls.html#using-sys-executable-to-spawn-subprocesses-that-outlive-the-application-process-implementing-application-restart

Marlon2025c commented 2 months ago

Thank you so much! After adding that to my code, it works perfectly. The issue with the modules and imports is completely resolved. I really appreciate your help!