search

Nuitka / Nuitka

Nuitka is a Python compiler written in Python. It's fully compatible with Python 2.6, 2.7, 3.4-3.12. You feed it your Python app, it does a lot of clever things, and spits out an executable or extension module.
http://nuitka.net
Apache License 2.0
11.32k stars 608 forks source link

Nuitka 2.3.11 fails to build onefile .exe where it used to work flawlessly .. #2975

Closed seclorum closed 9 hours ago

seclorum commented 4 days ago

python -m nuitka --version

2.3.11 Commercial: None Python: 3.7.5 (tags/v3.7.5:5c02a39a0b, Oct 15 2019, 00:11:34) [MSC v.1916 64 bit (AMD64)] Flavor: Unknown Executable: C:\SDK\tools\pyenv37\Scripts\python.exe OS: Windows Arch: x86_64 WindowsRelease: 10


- How did you install Nuitka and Python

_The intention is to have a single "make.bat" file which prepares our tool (uartTOOL.py) for distribuation in the form of a single uartTOOL.exe executable.  Python is part of a vendor-supplied SDK toolkit, and Nuitka is installed with pip in an activated environment from the python toolkit._

Here is the contents of the make.bat, which includes the nuitka parameters used:

@echo off @echo "This will make the standalone version of the uartTOOL:" rem Always use locally-installed python instead of any system python call C:\SDK\python\tools\pyenv37\scripts\activate.bat pip install nuitka rem nuitka --windows-console-mode=disable --onefile --assume-yes-for-downloads --plugin-enable=tk-inter --remove-output .\uartTOOL.py nuitka --onefile --force-stdout-spec=uartTOOL.out.txt --force-stderr-spec=uartTOOL.err.txt --assume-yes-for-downloads --plugin-enable=tk-inter .\uartTOOL.py --output-filename=uartTOOL.exe dir *.exe


- The specific PyPI names and versions

  > python -m pip list -v

python -m pip list -v Package Version Location Installer

ACAT 1.12.4 c:\SDK\tools\pyenv37\lib\site-packages pip adk-workspace 1.1.0 c:\SDK\tools\pyenv37\lib\site-packages pip aptXOfflineDecoder 0.3.0 c:\SDK\tools\pyenv37\lib\site-packages pip attrs 23.2.0 c:\SDK\tools\pyenv37\lib\site-packages pip backcall 0.2.0 c:\SDK\tools\pyenv37\lib\site-packages pip cachetools 5.3.3 c:\SDK\tools\pyenv37\lib\site-packages pip colorama 0.4.6 c:\SDK\tools\pyenv37\lib\site-packages pip contextlib2 21.6.0 c:\SDK\tools\pyenv37\lib\site-packages pip customer-pydbg-base 2.24.1 c:\SDK\tools\pyenv37\lib\site-packages pip customer-pydbg-cextn 1.4.2 c:\SDK\tools\pyenv37\lib\site-packages pip customer-pydbg-cryptodll 1.0.0 c:\SDK\tools\pyenv37\lib\site-packages pip customer-pydbg-hydratrans 1.2.0 c:\SDK\tools\pyenv37\lib\site-packages pip customer-pydbg-qcc514x-qcc304x 2.24.1 c:\SDK\tools\pyenv37\lib\site-packages pip customer-pydbg-qcc516x7x-qcc306x7x 2.24.1 c:\SDK\tools\pyenv37\lib\site-packages pip cx-Freeze 6.15.16 c:\SDK\tools\pyenv37\lib\site-packages pip cx-Logging 3.1.0 c:\SDK\tools\pyenv37\lib\site-packages pip cycler 0.11.0 c:\SDK\tools\pyenv37\lib\site-packages pip Cython 3.0.10 c:\SDK\tools\pyenv37\lib\site-packages pip decorator 5.1.1 c:\SDK\tools\pyenv37\lib\site-packages pip fonttools 4.38.0 c:\SDK\tools\pyenv37\lib\site-packages pip graphviz 0.8.4 c:\SDK\tools\pyenv37\lib\site-packages pip importlib-metadata 6.7.0 c:\SDK\tools\pyenv37\lib\site-packages pip inflection 0.5.1 c:\SDK\tools\pyenv37\lib\site-packages pip ipython 7.34.0 c:\SDK\tools\pyenv37\lib\site-packages pip jedi 0.19.1 c:\SDK\tools\pyenv37\lib\site-packages pip Jinja2 3.1.2 c:\SDK\tools\pyenv37\lib\site-packages pip jsonschema 3.2.0 c:\SDK\tools\pyenv37\lib\site-packages pip kal-python-tools 1.1.12 c:\SDK\tools\pyenv37\lib\site-packages pip kalcmd2 2.8.2 c:\SDK\tools\pyenv37\lib\site-packages pip kiwisolver 1.4.5 c:\SDK\tools\pyenv37\lib\site-packages pip kse 0.30.0 c:\SDK\tools\pyenv37\lib\site-packages pip KSP 0.6.2 c:\SDK\tools\pyenv37\lib\site-packages pip lief 0.12.3 c:\SDK\tools\pyenv37\lib\site-packages pip MarkupSafe 2.1.3 c:\SDK\tools\pyenv37\lib\site-packages pip matplotlib 3.5.3 c:\SDK\tools\pyenv37\lib\site-packages pip matplotlib-inline 0.1.6 c:\SDK\tools\pyenv37\lib\site-packages pip multi-coredump 0.1.9 c:\SDK\tools\pyenv37\lib\site-packages pip Nuitka 2.3.11 c:\SDK\tools\pyenv37\lib\site-packages pip numpy 1.21.6 c:\SDK\tools\pyenv37\lib\site-packages pip ordered-set 4.1.0 c:\SDK\tools\pyenv37\lib\site-packages pip packaging 23.2 c:\SDK\tools\pyenv37\lib\site-packages pip parso 0.8.3 c:\SDK\tools\pyenv37\lib\site-packages pip pefile 2023.2.7 c:\SDK\tools\pyenv37\lib\site-packages pip pickleshare 0.7.5 c:\SDK\tools\pyenv37\lib\site-packages pip Pillow 9.5.0 c:\SDK\tools\pyenv37\lib\site-packages pip pip 24.0 c:\SDK\tools\pyenv37\lib\site-packages pip prompt-toolkit 3.0.43 c:\SDK\tools\pyenv37\lib\site-packages pip psutil 5.6.2 c:\SDK\tools\pyenv37\lib\site-packages pip py-genparse-at 1.1.1 c:\SDK\tools\pyenv37\lib\site-packages pip py2exe 0.12.0.2 c:\SDK\tools\pyenv37\lib\site-packages pip pydbg-default 2.24.1 c:\SDK\tools\pyenv37\lib\site-packages pip Pygments 2.17.2 c:\SDK\tools\pyenv37\lib\site-packages pip pyparsing 3.1.1 c:\SDK\tools\pyenv37\lib\site-packages pip pyreadline 2.1 c:\SDK\tools\pyenv37\lib\site-packages pip pyrsistent 0.19.3 c:\SDK\tools\pyenv37\lib\site-packages pip pyserial 3.4 c:\SDK\tools\pyenv37\lib\site-packages pip PySide2 5.13.1 c:\SDK\tools\pyenv37\lib\site-packages pip python-dateutil 2.8.2 c:\SDK\tools\pyenv37\lib\site-packages pip pywin32 224 c:\SDK\tools\pyenv37\lib\site-packages pip PyYAML 6.0.1 c:\SDK\tools\pyenv37\lib\site-packages pip qcom-ubuild 1.2.4 c:\SDK\tools\pyenv37\lib\site-packages pip Qualcomm-TestAG 1.0.3 c:\SDK\tools\pyenv37\lib\site-packages pip schema 0.7.5 c:\SDK\tools\pyenv37\lib\site-packages pip SCons 4.4.0 c:\SDK\tools\pyenv37\lib\site-packages pip SConsVariants 0.2.20 c:\SDK\tools\pyenv37\lib\site-packages pip setuptools 68.0.0 c:\SDK\tools\pyenv37\lib\site-packages pip shiboken2 5.13.1 c:\SDK\tools\pyenv37\lib\site-packages pip six 1.16.0 c:\SDK\tools\pyenv37\lib\site-packages pip traitlets 5.9.0 c:\SDK\tools\pyenv37\lib\site-packages pip trbtrans 2.11.0 c:\SDK\tools\pyenv37\lib\site-packages pip typing_extensions 4.7.1 c:\SDK\tools\pyenv37\lib\site-packages pip wcwidth 0.2.13 c:\SDK\tools\pyenv37\lib\site-packages pip wheel 0.41.3 c:\SDK\tools\pyenv37\lib\site-packages pip wrapt 1.16.0 c:\SDK\tools\pyenv37\lib\site-packages pip zipp 3.15.0 c:\SDK\tools\pyenv37\lib\site-packages pip zstandard 0.21.0 c:\SDK\tools\pyenv37\lib\site-packages pip Link requires a different Python (3.7.5 not in: '>=3.8'): https://files.pythonhosted.org/packages/1e/65/22725f8ba583376d0c300c3b9b52b9a67cfd93d786a80be73c167e45abc8/pip-24.1b1-py3-none-any.whl (from https://pypi.org/simple/pip/) (requires-python:>=3.8) Link requires a different Python (3.7.5 not in: '>=3.8'): https://files.pythonhosted.org/packages/71/38/b0cb3d68b4776b6208a2f16b6d444a848a1fe465a78ce4b7dbbeb8a4fc58/pip-24.1b1.tar.gz (from https://pypi.org/simple/pip/) (requires-python:>=3.8) Link requires a different Python (3.7.5 not in: '>=3.8'): https://files.pythonhosted.org/packages/c1/64/0f1528ed3dfd75a56e084caab3c8dbed596bf87a1dfc40cbc57166bb0c25/pip-24.1b2-py3-none-any.whl (from https://pypi.org/simple/pip/) (requires-python:>=3.8) Link requires a different Python (3.7.5 not in: '>=3.8'): https://files.pythonhosted.org/packages/0b/b2/1d1eac32a16fd9478fb9bc6eb6b899f91e6fecceba194fa097c35d385e9a/pip-24.1b2.tar.gz (from https://pypi.org/simple/pip/) (requires-python:>=3.8) Link requires a different Python (3.7.5 not in: '>=3.8'): https://files.pythonhosted.org/packages/25/49/2255373efd193c6fbd97dc22399e9c830a6517a0f02ca77fbc0bd83ac5cc/pip-24.1-py3-none-any.whl (from https://pypi.org/simple/pip/) (requires-python:>=3.8) Link requires a different Python (3.7.5 not in: '>=3.8'): https://files.pythonhosted.org/packages/df/60/7538a9cd4d76829c2f585b73d6e400c833a1769408451c813354c58de6cc/pip-24.1.tar.gz (from https://pypi.org/simple/pip/) (requires-python:>=3.8) Link requires a different Python (3.7.5 not in: '>=3.8'): https://files.pythonhosted.org/packages/f4/ab/e3c039b5ddba9335bd8f82d599eb310de1d2a2db0411b8d804d507405c74/pip-24.1.1-py3-none-any.whl (from https://pypi.org/simple/pip/) (requires-python:>=3.8) Link requires a different Python (3.7.5 not in: '>=3.8'): https://files.pythonhosted.org/packages/c0/d0/9641dc7b05877874c6418f8034ddefc809495e65caa14d38c7551cd114bb/pip-24.1.1.tar.gz (from https://pypi.org/simple/pip/) (requires-python:>=3.8)


- Note if this is a regression

_Was working fine (generating the .exe) with Nuitka-2.3 (not 2.3.11)

Error:

Collecting nuitka Downloading Nuitka-2.3.11.tar.gz (3.7 MB) ---------------------------------------- 3.7/3.7 MB 10.3 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: ordered-set>=4.1.0 in c:\SDK\python\tools\pyenv37\lib\site-packages (from nuitka) (4.1.0) Requirement already satisfied: zstandard>=0.15 in c:\SDK\python\tools\pyenv37\lib\site-packages (from nuitka) (0.21.0) Building wheels for collected packages: nuitka Building wheel for nuitka (pyproject.toml) ... done Created wheel for nuitka: filename=Nuitka-2.3.11-cp37-cp37m-win_amd64.whl size=3383741 sha256=f4da42bea606c8e65efdfaa0f8701d27d51d1553bb8c0e587305f8f1bd0c0185 Stored in directory: c:\users\uartBUILDER\appdata\local\pip\cache\wheels\23\2b\43\f5e575b7b07222ded85cf1a658271a2a80c9e4c6b0ea652189 Successfully built nuitka Installing collected packages: nuitka Successfully installed nuitka-2.3.11 Nuitka-Options: Used command line options: --onefile --force-stdout-spec=uartTOOL.out.txt --force-stderr-spec=uartTOOL.err.txt --assume-yes-for-downloads Nuitka-Options: --plugin-enable=tk-inter .\uartTOOL.py --output-filename=uartTOOL.exe Nuitka:WARNING: On Windows, support for input/output on the console Windows, does not work on non-UTF8 systems, unless Python 3.8 or higher is used but this is 3.7, so please consider Nuitka:WARNING: upgrading, or disabling the console window for deployment. Nuitka:WARNING: Complex topic! More information can be found at https://nuitka.net/info/old-python-windows-console.html Nuitka: Starting Python compilation with Nuitka '2.3.11' on Python '3.7' commercial grade 'not installed'. Nuitka: Completed Python level compilation and optimization. Nuitka: Generating source code for C backend compiler. Nuitka: Running data composer tool for optimal constant value handling. Nuitka: Running C compilation via Scons. Nuitka-Scons: Backend C compiler: cl (cl 14.3). Nuitka-Scons: Backend linking program with 14 files (no progress information available for this stage). Nuitka-Scons: Compiled 14 C files using clcache with 1 cache hits and 13 cache misses. Nuitka-Plugins:tk-inter: Included 87 data files due to Tk needed for tkinter usage. Nuitka-Plugins:tk-inter: Included 830 data files due to Tcl needed for tkinter usage. Nuitka-Postprocessing: Creating single file from dist folder, this may take a while. Nuitka-Onefile: Running bootstrap binary compilation via Scons. Nuitka-Scons: Onefile C compiler: cl (cl 14.3). Nuitka-Scons: Onefile linking program with 1 files (no progress information available for this stage). Nuitka-Scons: Compiled 1 C files using clcache with 1 cache hits and 0 cache misses. Nuitka-Onefile: Using compression for onefile payload. Traceback (most recent call last): File "C:\SDK\tools\pyenv37\lib\site-packages\nuitka__main.py", line 161, in main() File "C:\SDK\tools\pyenv37\lib\site-packages\nuitka\main__.py", line 147, in main MainControl.main() File "C:\SDK\tools\pyenv37\lib\site-packages\nuitka\MainControl.py", line 1130, in main _main() File "C:\SDK\tools\pyenv37\lib\site-packages\nuitka\MainControl.py", line 1043, in _main packDistFolderToOnefile(dist_dir) File "C:\SDK\tools\pyenv37\lib\site-packages\nuitka\freezer\Onefile.py", line 52, in packDistFolderToOnefile packDistFolderToOnefileBootstrap(onefile_output_filename, dist_dir) File "C:\SDK\tools\pyenv37\lib\site-packages\nuitka\freezer\Onefile.py", line 243, in packDistFolderToOnefileBootstrap start_binary=getResultFullpath(onefile=False), File "C:\SDK\tools\pyenv37\lib\site-packages\nuitka\freezer\Onefile.py", line 150, in runOnefileCompressor low_memory=Options.isLowMemory(), File "C:\SDK\tools\pyenv37\lib\site-packages\nuitka\tools\onefile_compressor\OnefileCompressor.py", line 326, in attachOnefilePayload _attachOnefilePayload() File "C:\SDK\tools\pyenv37\lib\site-packages\nuitka\utils\Utils.py", line 395, in retryingFunction result = func(*args, **kwargs) File "C:\SDK\tools\pyenv37\lib\site-packages\nuitka\tools\onefile_compressor\OnefileCompressor.py", line 240, in _attachOnefilePayload file_list.remove(start_binary) ValueError: list.remove(x): x not in list Nuitka-Reports: Compilation crash report written to file 'nuitka-crash-report.xml'. Please include it in your bug report.

Dnyarri commented 4 days ago

Just a guess but what happen if you add --onefile-no-compression?

gdementen commented 2 days ago

Please supply a "Short, Self Contained, Correct, Example" of code reproducing the error. It is obviously related to onefile mode, but there is some other condition for triggering the problem which is unclear (at least for me) from reading the information you give.

Without that, it is very hard to reproduce your problem and makes fixing it much harder.

Providing the compilation crash report ('nuitka-crash-report.xml') as asked by the program would also help.

kayhayen commented 9 hours ago

So, based on the call stack, --output-filename plays a role here, that I couldn't really figure out, what the issue could be with it. I simply will change the removal to one that compares filenames harder.

Since nothing has forthcome by the reporter, I am closing this early and will not trace the changes here.