Smoother dependency install process

[justinpenner]

Installing dependencies for Glyphs scripts and plugins is tricky. TalkingLeaves requires a few Python libraries (hyperglot, urlreader, vanilla, objc), but installing and managing these modules is complicated due to the following factors:

• Glyphs uses its own custom "GlyphsPython" build by default
• Glyphs users can switch to another Python on their system via the preferences window
• Python looks for imports in sys.path list of paths, which changes depending which Python the user selected in preferences
• Glyphs users can install certain modules (vanilla, fontTools, etc) from the Plugin Manager window. These are installed in the Repositories folder, and an alias is added to the Scripts folder so they can be found by Python. This location is near the top of sys.path, which means modules installed this way cannot be overridden by other versions installed by other Pythons, or by packages installed in Scripts/site-packages
• GlyphsPython comes with its own copy of pyobjc preinstalled, but its location is at the end of the sys.path list, so it can be overridden by pyobjc in other locations
• Glyphs team recommends installing packages in ~/Application Support/Glyphs 3/Scripts/site-packages, but this location was only added to sys.path from 3.2, so it needs to be manually added to sys.path for Glyphs <3.2, and it only works if GlyphsPython is being used
• Plugins get added to the top of sys.path, so plugins cannot bundle their own private modules without the risk of interfering with other plugins that may require a different version of the same module.
• The Plugin Manager window automatically installs certain required modules, including hidden ones that certain plugins require, but all of these modules are free from dependencies, so I don't think modules requiring dependencies can be added there.

What all of that boils down to is: Glyphs gives developers two recommended methods for installing packages (Scripts/site-packages and the Plugin Manager). The former method is less user-friendly and has potential for conflicts between plugins. The latter forces plugins to use the same version of modules, eliminating conflicts but also eliminating the developer's choice of module versions. The latter also forces important modules like Vanilla to be beholden to the Glyphs Vanilla fork, which is not updated when the upstream Vanilla is updated.

So, the only package management system I can imagine that solves all these problems, is a custom one that allows TalkingLeaves to install and use any packages, but keeps them private from other plugins. I suspect this could be done in some way, but I need to test some ideas.

[justinpenner]

I tested some things. I think I could write a script, call it gpip.py and drop it into the plugin's Resources folder. The script would be a module that searches for any pip that's already installed on the system, and uses it to install the necessary modules, targeting them to a private folder like Plugins/TalkingLeaves-packages/_tl. Then, if I append Plugins/TalkingLeaves-packages to sys.path, the plugin will be able to use a private prefix like import _tl.hyperglot as hyperglot.

This keeps the dependencies separate from other plugins, and allows me to automate the package install process. I've tested it with different pips in various locations on my system and it doesn't seem to matter what Python version number the pip is from, since it's effectively just putting the packages in the target location, and the packages are the same no matter which pip I use.

[justinpenner]

The prefix idea didn't work, because it prevents the modules from being able to import anything. I should have thought of that. So, I don't think there's any way to make modules even pseudo-private to a plugin, without touching the code inside of the modules. I'm now working under the assumption that all plugins are forced to share common modules.

I've run into a number of roadblocks trying to install dependencies via pip. The main issue is that I cannot get pip to work within GlyphsPython, unless the user has installed the same version of Python from Python.org, which is too much of a burden for users.

Some things I've tried:

• Running GlyphsPython's pip from the command line. All the Python and pip executables unfortunately do not work, as they all try to point to /Library/Frameworks/Python.framework/… which is where Python.org installs Python. I can't find any working Python executable in GlyphsPython
• Running pip via runpy in the Macro window. It complains that setuptools is missing. Indeed, setuptools doesn't seem to be present anywhere in GlyphsPython.
• Running ensurepip in the Macro window. It also complains about a missing setuptools wheel. If I download that exact wheel from pypi.org, Glyphs gets a spinning wheel and stops responding.
• Running pip from the macOS system Python. This works for some packages but not others since it's a different Python version number.

[justinpenner]

I wasn't able to find a simple way to fix pip in GlyphsPython, because it seemed like it would involve editing or rebuilding some Python binaries which is far beyond my knowledge. Georg at Glyphs mentioned he'll try to get it working, but in the meantime the workaround must be for users to install their own Python as a pre-requisite for TalkingLeaves. Install instructions in README.md have been updated to make it as easy as possible.

Closing for now, as we now have the best available dependency install process, even if it isn't ideal. Will reopen this issue if anything changes.

[justinpenner]

Before adding TalkingLeaves to the Plugin Manager, I'd like to automate the dependency installs as much as possible.

• [ ] Mention Python requirement in Plugin Manager description
• [ ] Check if any dependencies are missing each time plugin starts
• [ ] Move the pip install command from README to an automated async subprocess call
• [ ] Look for pip in common paths, otherwise open a dialog prompting the user to install Python or tell us where pip is located
• [ ] Dialog asking the user to confirm the dependency installs