standalone fontmake executable

[anthrotype]

Some of you may have noticed that in the latest fontmake release v1.2.3, I uploaded a stand-alone fontmake executable which bundles everything in a single file and does not even require Python to be installed.

I used the latest pyinstaller and Python 3.6.1 (official builds from Python.org) from my local machine, with OS X 10.10 and Windows 7 (64-bit).

The two zip files simply contain fontmake (or fontmake.exe on Windows) plus the LICENSE file. One can simply download it, unzip and run it in place, or move it anywhere in the shell's $PATH, like /usr/local/bin.

E.g., from the Mac bash one could do this:

curl -LO https://github.com/googlei18n/fontmake/releases/download/v1.2.3/fontmake-standalone-1.2.3-py36-macosx.zip
unzip fontmake-standalone-1.2.3-py36-macosx.zip
./fontmake --help

I thought that this could maybe lower the entry barrier and make fontmake more easily accessible for non Python developers, or those are not familiar with pip and virtual environments, etc.

If there's enough interest, I could set up Travis and Appveyor to automatically build the standalone executable upon tagged commits.

Note this is still experimental as I haven't tested it extensively, so I invite anyone who's interested in trying this out and report any issues.

Please let me know what you think. Thanks

[punchcutter]

I tried out the stand-alone today and it failed here because Python 3:

Building variations tables
Traceback (most recent call last):
  File "fontmake/__main__.py", line 198, in <module>
  File "fontmake/__main__.py", line 182, in main
  File "fontmake/font_project.py", line 352, in run_from_glyphs
  File "fontmake/font_project.py", line 408, in run_from_designspace
  File "fontmake/font_project.py", line 458, in run_from_ufos
  File "fontmake/font_project.py", line 149, in build_otfs
  File "fontTools/misc/loggingTools.py", line 372, in wrapper
  File "fontmake/font_project.py", line 281, in save_otfs
  File "fontmake/font_project.py", line 550, in _closest_location
TypeError: 'dict_keys' object does not support indexing
Failed to execute script __main__

[anthrotype]

Thanks Zachary. That looks like a bug in the actual code, rather than an issue in the standalone wrapper. I'll fix that.

[anthrotype]

I just tried using a tool called pex that can create self-contained executable environments that only require python to run. Currently this only works on Linux and macOS (where some version of python is always preinstalled). If you wish to try this out, download this zip file, extract the file fontmake.pex contained inside, and then simply run it as if it were an executable shell script.

fontmake-1.4.0.pex.zip

$ ./fontmake.pex --help

The pex file is really a zip file with a shebang and some bootstrapping code that finds the right python interpreter and runs the embedded entry point, importing the required libraries as embedded wheel files. I think it's really cool. It's also super easy to build. I may set up Travis to create one and upload it to the Github Releases page for the next version of fontmake.

WDYT?

[anthrotype]

By default it will try to run fontmake with the first compatible python that it finds in your environment (this particular fontmake.pex I made supports both python2.7 and python3.6). To run it with a specific interpreter version, you can export a PEX_PYTHON=python3.6 environment variable before running ./fontmake.pex. The .pex file extension is not required. One can even move it to a bin folder in one's $PATH and run it as simply fontmake, as if it were a native executable.

You can even fire an interactive python shell with the env variable PEX_INTERPRETER=1 ./fontmake.pex!

[miguelsousa]

I thought that this could maybe lower the entry barrier and make fontmake more easily accessible for non Python developers, or those are not familiar with pip and virtual environments, etc.

The AFDKO used to be self-contained similarly to what you're trying to do, and we've abandoned that idea. In general I understand the motivation of making tools easier to use, but it usually adds unnecessary complexity and maintenance headaches. fontmake is documented well enough for anyone to use it as is, if he/she is really committed.

[anthrotype]

This is different because it doesn’t include a full python runtime like the afdko used to do. Think of it as a zipped virtualenv that can be run like a script with zero configuration, no need to install pip or virtualenv. Anyway, just an idea.

[behdad]

I will be using something like this. Updating my env every time to roll fontmake forward is a pain.

[anthrotype]

Of course, I’ll always keep uploading fontmake to PyPI, but this could be an alternative way of deployment.

Btw, pex was developed by a guy at Twitter, and it’s a mature project, they’ve been using it in production for some time now. Here’s brief introduction video: https://youtu.be/NmpnGhRwsu0

[chrissimpkins]

I really like this approach Cosimo. The ability to pull a pre-built *.pex executable from your releases would be helpful to us. +1

[chrissimpkins]

If you do push it as a release, would it be possible to make separate Py3 and Py2 versions instead of one that defaults to either interpreter? Or Py3 only?

[anthrotype]

Glad you like it.

would it be possible to make separate Py3 and Py2 versions instead of one that defaults to either interpreter?

sure, like these:

fontmake-1.4.0-py2.pex.zip

fontmake-1.4.0-py3.pex.zip

(though I kind of like the fact that one can do one a single pex with both p2 and py3 wheels and lets the user choose which one to use)

[chrissimpkins]

Does py2 use shebang for env python and py3 use env python3 or does it specify exact py2 and py3 minor versions? Trying to understand whether the use of the pex without an env variable set breaks when system installed Py3 versions change.

[anthrotype]

I used python2.7 and python3.6 for the shebangs of those two. I could in theory make a pex that has both python3.5 and python3.6 wheels. But yes, every time there’s a new minor python version (e.g. 3.7) the C extension modules (in our case pyclipper and compreffor) need to be recompiled.

[chrissimpkins]

Thanks Cosimo.

[anthrotype]

https://github.com/googlei18n/fontmake/releases/tag/v1.7.2

I just added self-contained python zipapps for all the platforms/python versions that are currently supported by fontmake. Just download the fontmake*.pyz file that matches your python, platform and architecture and run it like a normal executable. Yes, a python installation is still requireed. However, there is no longer need to create a virtual environments or install any extra dependencies. They are all embedded in the .pyz file which works like a native executable (it is actually a zip file with some magic bootstrapping python code and the whole site-packages embedded). Please report any issues that you find.

If you're curious, these were created using https://github.com/linkedin/shiv

Ah, for some reason the upload from Travis CI failed (maybe they are too many or too big), so I created them locally and uploaded them manually. I'll try to automate this better another time.

FYI I'll be away until the 1st of September. Cheers

[anthrotype]

on Unix, you can rename the .pyz files to just fontmake if you like, they will just work. On windows, you need to keep the .pyz extension (so you can rename it fontmake.pyz), this is so that the py launcher that comes with Python on Windows is able to recognize the zipapp (it is registered to run files with those extensions) and can correctly resolve the shebang. I called the .pyz files with the long explicit names using the same naming scheme used by python wheels in order to better distinguish them.

[anthrotype]

I just realised that the fontmake.pyz Python zipapps that I published for v1.7.2 only work with Python 3. Also, it appears the unix file executable flag is missing so one needs to run chmod +x fontmake.pyz before being able to run the downloaded zipapp. I didn't even bother testing that they worked, sorry ;)

I just updated the build script so that it packages up the fontmake executable inside a zip archive, so we can include a LICENSE file (maybe in the future a README), and the zipped executable can be named just fontmake and keep the execution flag.

You'll find them at the same download link: https://github.com/googlei18n/fontmake/releases/tag/v1.7.2

I would like to hear from you if publishing a self-contained fontmake is worth the effort.

This fontmake.pyz solution proposed here still requires that the user has at least Python 3 installed. For now I only published pyz files for python 3.6 and 3.7 (for all three platforms, well, 4 if you count 32 vs 64-bit on windows).

The other solution was to use pyinstaller and create a truly self-contained app that doesn't even require Python to be installed.

My current preference currently leans towards the .pyz method:

1) installing python 3 is easy, there are installers from python.org, and many user guides available 2) there are high chances that python 3 is already installed on a font developer's machine, since most of the other tools are also python based.

[anthrotype]

I wonder.. is anybody finding these "zipapp" version of fontmake useful at all?

[chrissimpkins]

I haven't had an opportunity to try them yet Cosimo. I recently built pyinstaller versions (on Py3.6) of fontmake based on my Python script that allows us to compile UFO sources in parallel. That is likely where we are headed with the compile. Haven't tested it yet. I like the self-contained Python executable approach much more than venv's...

[twardoch]

I think it works be cooler to make a "fonttools make" thin command or another such "virtual" tool that has subcommands, and package that. That could in fact be a merger of fonttools, fontmake, afdko and perhaps a few other useful ones.

Call it "open fonttools", oft in short. So the user gets one file like "oft" or "oft.exe" and can run "oft psautohint ..." "oft subset ..." "oft fontmake ..." and so on.

Just fontmake is "not attractive enough". fontbakery has subcommands which makes is useful. I have a feeling that cli with subcommands are trendy these days (and so are cli apps that take yaml config files as alternative argument lists).

[twardoch]

I made a little pseudopackage (not in any way like this, super primitive): https://github.com/twardoch/fontsurgery-tools/blob/master/README.md

It also installs ttfautohint and the harfbuzz commandline tools. Ideally, those could also be included transparently.

Then it'd be really handy — one file that requires no install, might or might not package the Python interpreter depending on the OS, exists in a separate repo and is this kind of cli shell for font work.

Right now because of the py2/3 dilemma/transition (real) — whenever I install some python package, I do

for pip in pip2 pip3; do $pip install --user --upgrade afdko; done;

but that's ridiculous :)

So, yes for a one-file commandline thing but one that brings some real value and useful subtools, and radically simplifies the lives for people who are ok with simple commandline but freaking with pip2/pip3 and other package maintenance is too much.

[chrissimpkins]

@twardoch The one tool to rule them all :)

@anthrotype Any thoughts about font editor add ons or a GUI tool for fontmake Cosimo? That may open up the fontmake user base in a way that no CLI tool will adequately address. Some just aren't comfortable on that terrain no matter how simple you make the installation approach.

[anthrotype]

I decided to no longer build and publish fontmake's standalone zip-apps in the Github Releases page, as they don't seem to be used much, as far as I can tell. If you want to unstall fontmake in an isolated environement (without the hassle of manually maintaining virtual environments), I recommend that you use pipx. After you have installed pipx following https://pipxproject.github.io/pipx/installation/, you can then do pipx install fontmake and pipx upgrade fontmake (or pipx upgrade-all).