No module named tkinter

[ksamuel]

When using pyflow on Windows 10, if I ask it to use Python 3.7 but haven't it installed, pyflow will install it for me. Unfortunatly it will install it without the tkinter module, which is normally installed with the standard Python installer.

[David-OConnor]

This is a known bug with the compiled versions of Python pyflow uses. Variations of it exist on both the Windows and Linux binaries. It's technically not something that would be fixed in this repo, but by the companion repo that has these binaries.

Basically, if you use the default compiling instructions python.org provides, it's missing some things that you'd expect to be installed, like tkinter.

The workaround, as you probably are aware, is to install Python 3.7 using an official installer. However, this is something I want Pyflow to handle. I need to research more on how to built Python with the expected extras. Let me know if you have any ideas or experience with this.

[ksamuel]

One possibility is to actually use the official Python installer. Indeed, there are several options that let you control the process, making it silent, install in a particular directory, etc.

You can see an example and a list of all the arguments available here: https://eddiejackson.net/wp/?p=10276

I think the biggest challenge is to chose the proper options, and then find a way to avoid the py launcher listing the newly installed python. I think the py launcher create its listing by using PEP514 (https://www.python.org/dev/peps/pep-0514/), but I can't find a flag to say to the installer "don't write to the register", and deleting the entry after the fact is probably not great.

[ksamuel]

As I suspected, there is no flag to avoid the register entry, but there is one trick: you can install python in a temp dir, copy the files where you want them, then uninstall python.

[David-OConnor]

The python-installer trick sounds dirty, but... It might actually work. Or, if I can extract the files it installs into a portable form - that would be better. I wish Python.org would release portable binaries, or provide instructions on how to build with the standard features. I'm not great at C compiling and linking, so this might be a good workaround.

[ksamuel]

Indeed, it would be cleaner. Especially because if anything interrupt the process, it may result in a half installed python.

An alternative would be to do that on one machine once, copy the files, zip them, and host them, then have pyflow dl them from there.

Of course this means doing that for each new python release, or setuping a system to do it automatically.

But that would be a cool service to setup as portable Python are usually very bloated: https://portablepython.com/

[David-OConnor]

I tried your install + copy folder + zip on Windows. It seems to work as standalone (Although I haven't tried on a diff computer etc). The resulting zip's 230mb. Yikes.

[ksamuel]

This is what a user would get if they download the exe from python.org and installed python from there. Not sure you can do better than the official blessed way.

Fortunately this should only be paid once per Python version: venv created from this are mostly symlink, no? On linux they are, but not sure on Windows.

[David-OConnor]

That's correct. Might still be the way to go.

[David-OConnor]

Got it down to 135mb using tar.xz, which is how the existing ones are. The old one was 8mb, but I guess this is the tradeoff for having the tools Python's expected to have.

At risk of being cavalier, it's now live, and doesn't require a new version of pyflow. This will only affect Windows users, and will hopefully solve these issues.

Will require a pyflow clear command followed by 4 to remove the old portable version.

[ksamuel]

pyflow being a very new tool, it's good to push things quickly at first. Nobody is building critical infra with it yet, and this way we can give feedback quickly. I'm gonna fire a VM right now so that you get a data point.

[ksamuel]

Took longer than expected, Ubuntu Secure boot now wants virtualbox kernel modules to be signed, so I went for dual boot instead.

I tried it with Python 3.8 on windows 10, and it worked. It was a bit long, as to be expected, so it may need a warning, and a progress bar, to avoid frustrating unexperienced users.

However, I tried with 3.6, and it didn't work. Steps:

pyflow new hop # chose python 3.6
cd hop
pyflow clear
pyflow install flask
pyflow python

But "import tkinter" still fails. The see the a "C:\Users\user\AppData\Roaming\pyflow\python-3.6.8-windows.tar.xz" file but it's 8mb.

Maybe some HTTP/DNS cache on my side?

[David-OConnor]

Thank you very much for testing! I only uploaded the 3.8 version, which would explain that. Agree on the progress bar. Wasn't needed before since the DL was so small.

I was able to test on a separate laptop on mine as well, so that's 3 data points this approach works. (Although I still couldn't get it to work with that pywin32/win32api issue someone posted.

Things to do:

• Build older versions of Python on Windows
• Sort it out for Linux (Probably with 2 versions: Newer (Ubuntu, Debian etc), and older (RH/Centos etc)
• Someone with a Mac would need to handle Mac support.
• Remove some of the bandaid code I've put in as workarounds, eg installing wheel, reverting to sys python for certain installation steps etc.

edit: The Linux approach isn't obvious. I tried installing 3.8 using the DeadSnakes repo. It puts a binary in usr/bin/python3.8, and a big file dump in /usr/lib/python3.8, but I can'd find the folder that should have share, lib, include, and bin.

[ksamuel]

Linux is going to be complicated: the combinations of distros + versions will make things difficult. E.G: deadsnake works only for Ubuntu LTS.

Several possible strats:

• make a mapping of all supported distros and their version, list the packages necessary to compile for each, and offer to install them, then download and compile python.
• statically compile each version of python, with all dependencies, and host that.
• host your own deb and rpm (and more?) repositories.
• use snap/flatpak.

There are a lot of pros and cons to each approach. "statically compile each version of python" is probably the most manageable on the long run, it can be automated, and I would guess the result would only depend of the version of libc. If you target an old one, it probably can work for many, many distro flavors. Kay Hayen (https://github.com/kayhayen), the author of nuitka (https://nuitka.net/) may have good advises on the topic.

There are also 2 projects that can help here:

• https://python-build-standalone.readthedocs.io
• https://pyoxidizer.readthedocs.io

The last one is espacially interesting given pyflow is already using rust.

[David-OConnor]

I think the approach I've been using is good-enough for supporting different distros: Packages that use C code, and Python installations seem to work if divided into the two broad categories I've listed above. Eg something built on CentOs7 works on CentOs, RedHat, and Fedora. Something built on Ubuntu 16.04 works on most distros that age and newer. (But something built on Ubuntu 20.04 won't work on 16.04; it's a one-way gate) It's certainly not guaranteed to work on an arbitrary Linux distro, but has worked on the ones I listed in the Readme, which includes the most popular ones.

Linux dudes often ask users to compile programs from source due to the problem you bring up, but I hate it; IME it never works right due to how fragmented the C build/linking ecosystem is.

Those projects you listed may be the best solution to this. Or, your bullet #2, for those 2 broad platforms.

[David-OConnor]

Updated old Windows versions.

[David-OConnor]

@ksamuel : Would you mind doing that same Windows check again with 3.6, now that I've updated the 3.6 and 3.7 binaries? I'm looking for another data point that this approach works. Thank you. (I've made no prog on the Linux front. I wonder if an another approach there is just figuring out what compiling options are needed to add the other stuff?)

[ksamuel]

Alright, I got the time to test them today. I could import tkinter 3.6 and 3.7 with no problem on windows 10.

Those tests are good candidates to run on MS Azur CI. Not only it would be automated (the free tier is generous, and they offer to run Windows, Mac and Linux VM), but you can trigger that from github actions.

What's more, that opens the door to much better testing. E.G: Downloading popular project, and running their unit tests so that you know your python are working ok.

EDIT: I had a doubt with 3.7, but it was my flaky connection playing tricks on me.

[David-OConnor]

Thank you for testing! Agree on the automated testing, although it's outside my current experience.

I'm going to leave this open, even though the exact issue you posted about is resolved, since similar ones are still present on Linux, and the fix is similar.