Nuitka / Nuitka

Nuitka is a Python compiler written in Python. It's fully compatible with Python 2.6, 2.7, 3.4-3.12. You feed it your Python app, it does a lot of clever things, and spits out an executable or extension module.
http://nuitka.net
Apache License 2.0
11.97k stars 643 forks source link

wirerope and methodtools cause nuitka binary to fail. #3145

Open eskiyerli opened 2 weeks ago

eskiyerli commented 2 weeks ago

python -m nuitka --version output:  1 ✘ 2.4 Commercial: None Python: 3.12.6 (main, Sep 8 2024, 13:18:56) [GCC 14.2.1 20240805] Flavor: Unknown Executable: /usr/bin/python OS: Linux Arch: x86_64 Distribution: Manjaro (based on arch) None Version C compiler: /usr/bin/gcc (gcc 14.2.1).

~/Downloads/RevEDA/reveda.dist /reveda.bin  ✔ Traceback (most recent call last): File "/home/eskiyerli/Downloads/RevEDA/reveda.dist/reveda.py", line 38, in File "", line 1360, in _find_and_load File "", line 1331, in _find_and_load_unlocked File "", line 935, in _load_unlocked File "/home/eskiyerli/Downloads/RevEDA/reveda.dist/revedaEditor/gui/revedaMain.py", line 43, in File "", line 1360, in _find_and_load File "", line 1331, in _find_and_load_unlocked File "", line 935, in _load_unlocked File "/home/eskiyerli/Downloads/RevEDA/reveda.dist/revedaEditor/backend/importViews.py", line 35, in File "", line 1360, in _find_and_load File "", line 1331, in _find_and_load_unlocked File "", line 935, in _load_unlocked File "/home/eskiyerli/Downloads/RevEDA/reveda.dist/revedaEditor/gui/libraryBrowser.py", line 46, in File "", line 1360, in _find_and_load File "", line 1331, in _find_and_load_unlocked File "", line 935, in _load_unlocked File "/home/eskiyerli/Downloads/RevEDA/reveda.dist/revedaEditor/gui/symbolEditor.py", line 42, in File "", line 1360, in _find_and_load File "", line 1331, in _find_and_load_unlocked File "", line 935, in _load_unlocked File "/home/eskiyerli/Downloads/RevEDA/reveda.dist/revedaEditor/gui/symbolScene.py", line 60, in File "", line 1360, in _find_and_load File "", line 1331, in _find_and_load_unlocked File "", line 935, in _load_unlocked File "/home/eskiyerli/Downloads/RevEDA/reveda.dist/revedaEditor/fileio/loadJSON.py", line 392, in File "/home/eskiyerli/Downloads/RevEDA/reveda.dist/revedaEditor/fileio/loadJSON.py", line 402, in PCellCache File "/home/eskiyerli/Downloads/RevEDA/reveda.dist/wirerope/rope.py", line 158, in call File "/home/eskiyerli/Downloads/RevEDA/reveda.dist/wirerope/callable.py", line 109, in init File "/home/eskiyerli/Downloads/RevEDA/reveda.dist/wirerope/callable.py", line 49, in detect_function_attr_name TypeError: function() missing required argument 'globals' (pos 2)

Offending code is:

@classmethod
@lru_cache(maxsize=100)
def getPCellDef(cls, file_path: str) -> dict:
    try:
        with open(file_path, "r") as temp:
            return json.load(temp)
    except (json.JSONDecodeError, FileNotFoundError):
        return {}

I have seen a similar issue report from 2020 but that was closed because the python and nuitka versions were very old. In this case, they are fairly recent. In this case, the versions are the latest: Python 3.12 methodtools 0.4.5 wirerope 0.4.5 Nuitka 2.4

kayhayen commented 2 weeks ago

Latest for Nuitka is 2.4.8 actually and esp. for 3.12 that might mean a lot. This also lacks a self contained example that I could compile, or so it seems.

eskiyerli commented 2 weeks ago

Same with Nuitka 2.4.8 and python 3.12 in a virtual environment. As it is a part of large software, I need to think of a self-standing example. The whole repository is in revedaRelease.

kayhayen commented 2 weeks ago

I cannot execute unknown and therefore untrusted code easily, so I don't compile whole projects outside of paid for contracts, but the stack trace should allow you to easily come up with something.

eskiyerli commented 2 weeks ago

In that case, here is a small code that uses methodtools with lru_cache. I ran it with Nuitka 2.4.8 and Python 3.12. It still has the same problem:


from methodtools import lru_cache

class DataProcessor:
    @classmethod
    @lru_cache(maxsize=100)
    def process_data(cls, data):
        # Simulate a time-consuming operation
        result = sum(int(x) for x in data if x.isdigit())
        print(f"Processing data: {data}")
        return result

    @classmethod
    def display_cache_info(cls):
        print(f"Cache info: {cls.process_data.cache_info()}")

if __name__ == "__main__":
    # First call - will process the data
    result1 = DataProcessor.process_data("123abc456")
    print(f"Result 1: {result1}")

    # Second call with the same input - will use cached result
    result2 = DataProcessor.process_data("123abc456")
    print(f"Result 2: {result2}")

    # Call with different input - will process new data
    result3 = DataProcessor.process_data("789xyz")
    print(f"Result 3: {result3}")

    # Display cache information
    DataProcessor.display_cache_info()
kayhayen commented 2 weeks ago

Seems wirerope does a bunch of is checks on the function type, that won't work the compiled functions that way. Replacing some of those seems easy to do. This one is more tricky:

    @cached_property
    def is_descriptor(self):
        if self.is_boundmethod:
            return False
        is_descriptor = type(self.wrapped_object).__get__ \
            is not types.FunctionType.__get__  # noqa
        if six.PY2:
            is_descriptor = is_descriptor and \
                not (self.is_unboundmethod or self.is_boundmethod)
        return is_descriptor

I tried editing it a bunch, but it seems very odd in its expectations of what can be done to a function.