Adjust for operation with Python 3.11 under Windows

[peter88213]

Actually, this is not a novelWriter bug. Anyway, I recommend to change the "shebang" from #!/usr/bin/env python3 to #!/usr/bin/python3 in order to avoid startup failures with the Python 3.11 launcher under Windows.

See: https://github.com/python/cpython/issues/100107

[vkbo]

Thanks for the heads up. This is really annoying since the recommended practice thus far has been the env method. I don't yet release novelWriter with 3.11 on Windows. Still on 3.10.

[vkbo]

I've been looking at this a bit. What is the use case here? Running novelWriter from source or from the installed version using the provided setup installer?

If it's the former, it requires changing the novelWriter.py file which is used for all platforms. In the latter case, it only involves modifying the generated noverlWriter.pyw file, which is simple enough to fix. However, in the latter case it is also designed to pick up the embedded pythonw.exe, not the system one.

[peter88213]

I think, this applies to running it from source, with a Python 3.11 installation from the python.org download page. I changed my own installation script to omit the shebang when building the startup script on Windows:

    #--- Create a start-up script.
    if platform.system() == 'Windows':
        shebang = ''
    else:
        shebang = '#!/usr/bin/python3\n'
    with open(f'{installDir}/{START_UP_SCRIPT}', 'w', encoding='utf-8') as f:
        f.write(f'{shebang}{START_UP_CODE}')

I do realize that this is not really necessary, because the new Python launcher works with #!/usr/bin/python3, but you never know what ideas the developers will come up with next when trying to emulate unix shell features under Windows.

[vkbo]

I'm not entirely sure what to do about this. I originally followed this recommendation when first moving beyond Linux for this project: https://stackoverflow.com/questions/6908143

Does it still launch with the old shebang if you call python novelWriter.py?

[peter88213]

Does it still launch with the old shebang if you call python novelWriter.py?

Yes, on the command line this seems to call directly the interpreter to which PATH leads. However, the point is to suppress the console window, so "pythonw" would be the better command. You can bypass the Python launcher in this way, but I have noticed that (for my novelyst program) starting by dragging and dropping a project file no longer works.

[vkbo]

Yes, I meant pythonw. Anyway, on the Windows installed version, I handle this by explicitly adding the pythonw.exe path bundled with novelWriter to the desktop and start menu icons. It passes novelWriter.pyw as a parameter instead.

https://github.com/vkbo/novelWriter/blob/66cc3777dc5072b5139013bfcd44e4edf5b83336/setup/win_setup_embed.iss#L48-L49

It was the only way I found to solve this without building an exe file of the project. I found that bundling everything in one large exe was both slow and sent half of the low end virus checkers panicking for no reason.

[peter88213]

On a side note, the .pyw extension seems to be meant as a discriminator for the Python launcher. If you call pythonw.exe directly, your script's extension may be .py as well.

I have not yet dealt with package building, which means my distribution comes as a zipfile with Python scripts. Some of my setup scripts create Windows registry files for context menu entries or file name associations. Here the absolute path to the interpreter is required, so I get it from sys.argv[0] of the setup script, which therefore must also have the .pyw extension.

[vkbo]

The .pyw extension does no harm though and ensures that you can also launch it by double-click on the launch script, so it makes somewhat sense still.

Anyway, my packaging solution is to just add the embedded python downloadable from python.org in the build folder with the novelwriter source, and then run pip on the requirements.txt file targeting a lib folder at the same location. Then I inject that into position 0 in sys.path so that I have my own little isolated environment.

The resulting package is installed into a folder under Program Files or in user space, and runs as a normal uncompressed python application with the latest versions of Python 3.10 and dependencies. It is significantly snappier than any other solutions I've tried.

[peter88213]

That reminds me of the cordless tools you can buy either withBattery and charger, or "barebone". However, the full Python distribution is quite a chunk, isn't it? I can't offer this when providing a whole bunch of different Python tools. What about PyInstaller? As far as I understand it, it packs only the required modules. And I don't mean the "All-in-one-exe" solution you mentioned.

[vkbo]

I think I've tried them all, and they randomly set off the low end virus software because they tend to fingerprint the installers rather than the malware that uses those installers. It's the lazy, cheap approach. I got a bit tired of users complaining about this. It is fully understandable that people are concerned when a virus scanner puts up an alert, even if it's a false alert. You can usually have it cleared up by contacting the vendors, but that is a massive annoyance for every release.

The bulky dependency for novelWriter is PyQt5. I actually have a script that deletes two thirds of it though, as I don't use any of the bulky components like the web engine and various embedded bits. Neither do I use the QtQuick framework. The Windows installer is down to about 30 MB, which is manageable.

In comparison, the AppImage release file is about 90 MB, and the Debian release a little over 2 MB, which is what novelWriter really boils down to.

[peter88213]

So let's assume there are several useful applications, all programmed with Python and built on top of QT. Then the central Python installation would probably be the most economical solution, but also the one that places the most constraints on the applications. Presumably, this would be the choice of users who know their system reasonably well. In any case, we would then have the use case to which the "shebang problem" refers, to return to the topic.

[vkbo]

Indeed. For packages it isn't really an issue, as rewriting the files for the Windows packages is not a big deal and can be done in the setup.py script. The issue is still whether going back to the old style shebang is right for Linux and Mac, in terms of running from the source code.

[peter88213]

If you leave out the shebang on Windows, it doesn't matter what it looks like for the other operating systems.