A critical error occurred when importing module nodes.nodes.image.image_file_iterator: module '__main__' has no attribute '__file__'

[nullmastermind]

Information:

• Chainner version: 0.16.0
• OS: Windows 11 Python 3.9.13

Description Error while I was trying to run yarn dev

Logs

[0] [2022-12-15 20:46:59 +0700] [17816] [ERROR] A critical error occurred when importing module nodes.nodes.image.image_file_iterator: module '__main__' has no attribute '__file__'
[0] [2022-12-15 20:46:59 +0700] [17816] [ERROR] A critical error occurred when importing module nodes.nodes.image.load_image: module '__main__' has no attribute '__file__'
[0] [2022-12-15 20:46:59 +0700] [17816] [ERROR] A critical error occurred when importing module nodes.nodes.image.paired_image_file_iterator: module '__main__' has no attribute '__file__'
[0] [2022-12-15 20:46:59 +0700] [17816] [ERROR] A critical error occurred when importing module nodes.nodes.image.save_image: module '__main__' has no attribute '__file__'

[joeyballentine]

Just to confirm, this is with running the current code in main?

[nullmastermind]

I pull master -> pip install -r requirement.txt -> yarn install -> yarn dev

Then I found the error, Load Image don't show

https://imgur.com/a/IcZyDAB

[joeyballentine]

Ive never seen this error before. @RunDevelopment @theflyingzamboni could this be related to the recent refactoring?

[theflyingzamboni]

I don't think so? I made sure that all of the nodes were appearing in the sidebar before the PR was merged, and one of the chains I tested used an image iterator and worked fine. I'm currently running dev, and have not had this issue.

[theflyingzamboni]

@bluevariant Maybe it's an issue with using yarn? Have you tried npm instead?

[joeyballentine]

idk why yarn would cause an issue since that's on the JS side. this is a python problem. But, it is a variable so it might be worth trying with npm

[theflyingzamboni]

Same, but it was something different.

[nullmastermind]

backend/src/nodes/impl/dds.py line 17

__TEXCONV_DIR = os.path.join(
    os.path.dirname(sys.modules["__main__"].__file__), "texconv"  # type: ignore
)

__file__ don't exist in multiprocessing

[joeyballentine]

@RunDevelopment

[RunDevelopment]

We don't use multiprocessing, so why does it not exist?

Also, I copied the sys.modules["__main__"].__file__ trick from add_caption. Any fix has to be applied there as well.

[nullmastermind]

Full log:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\root\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\root\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "C:\Users\root\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\root\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "C:\Users\root\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 288, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\root\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\root\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "D:\projects\Python\chaiNNer\backend\src\run.py", line 85, in <module>
    importlib.import_module(f"{module}", package=None)
  File "C:\Users\root\AppData\Local\Programs\Python\Python39\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "D:\projects\Python\chaiNNer\backend\src\nodes\nodes\image\image_file_iterator.py", line 11, in <module>
    from ..image.load_image import ImReadNode
  File "D:\projects\Python\chaiNNer\backend\src\nodes\nodes\image\load_image.py", line 17, in <module>
    from ...impl.DDS import dds_to_png_texconv
  File "D:\projects\Python\chaiNNer\backend\src\nodes\impl\dds.py", line 22, in <module>
    os.path.dirname(sys.modules["__main__"].__file__), "texconv"  # type: ignore
AttributeError: module '__main__' has no attribute '__file__'

I change dds.py to

try:
    main_script = sys.modules["__main__"].__file__
except:
    main_script = sys.argv[0]

__TEXCONV_DIR = os.path.join(os.path.dirname(main_script), "texconv")  # type: ignore
__TEXCONV_EXE = os.path.join(__TEXCONV_DIR, "texconv.exe")

work fine

[nullmastermind]

I don't know how the add caption trick works, but dds.py doesn't. Maybe something is wrong with runpy.

[RunDevelopment]

add_caption uses sys.modules["__main__"].__file__ within the function and not as a constant, so if an error occurs, you'll only see it when using the node. Could you please verify that this is the case?

---

Also, what is going on with your stack trace? From the stack trace, we can see that python starts our run.py in some multiprocessing manner. This is not okay since run.py starts a server on a specific port, and starting multiple servers listening to the same port obviously doesn't work.

This multiprocessing doesn't happen for me. Here is what my stack trace (using an artificial error) looks like:

File "./run.py", line 81, in <module>
    importlib.import_module(f"{module}", package=None)
File "C:\Python38\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "C:\Users\micha\Git\chaiNNer\backend\src\nodes\nodes\image\image_file_iterator.py", line 11, in <module>
    from ..image.load_image import ImReadNode
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "C:\Users\micha\Git\chaiNNer\backend\src\nodes\nodes\image\load_image.py", line 17, in <module>
    from ...impl.dds import dds_to_png_texconv
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "C:\Users\micha\Git\chaiNNer\backend\src\nodes\impl\dds.py", line 31, in <module>
    <the thing I used to cause an error>

So I don't understand why they are so different. Did you modify the dev npm script in any way?

[RunDevelopment]

I pull master -> pip install -r requirement.txt -> yarn install -> yarn dev

Then I found the error, Load Image don't show

https://imgur.com/a/IcZyDAB

I just noticed this. Your screenshot shows a custom node, so it's not an unmodified version of what's in main. Are you sure that the issue is not cause by any of your modifications?

Edit: I looked at your dev branch. Cool stuff, and I don't see why it would cause this issue. But I would still like to confirm that the issue also happens with an unmodified version.

[nullmastermind]

I pull master -> pip install -r requirement.txt -> yarn install -> yarn dev Then I found the error, Load Image don't show imgur.com/a/IcZyDAB

I just noticed this. Your screenshot shows a custom node, so it's not an unmodified version of what's in main. Are you sure that the issue is not cause by any of your modifications?

Edit: I looked at your dev branch. Cool stuff, and I don't see why it would cause this issue. But I would still like to confirm that the issue also happens with an unmodified version.

It happens on the unmodified version

[joeyballentine]

Did you try with npm instead of yarn?

[nullmastermind]

Did you try with npm instead of yarn?

I tried yarn, npm, pnpm & cd backend/src && python run.py 8000; I think the problem in Python

[nullmastermind]

dev branch

Maybe windows 11 is a problem. I'll try on windows 10

[joeyballentine]

I use windows 11 and this has never happened to me.

[green-s]

Ran into this error after upgrading to v0.17.0, using a system Python (3.10.8) with a few deps upgraded. Making a change to dds.py like the one above fixed it, and I didn't get any error when adding a caption. However downgrading Sanic to 21.9.3 fixes it without the change. I'm guessing it's related to the worker manager added in 22.9.

[RunDevelopment]

@joeyballentine Do we even support py 3.10 anymore? I believe I remember that there was an issue with this version.

[RunDevelopment]

The issue here was that sanic changed the default from multithreading to multiprocessing. We got the exact same error after upgrading sanic. We fixed this in #1695, so I will now close the issue.