Feature Request - Limit / Filter Characters

[diginfo]

For some fonts (especially big ones) often I do no require the full ASCII (127) Chars, but only require numeric and a few others i.e.

0 1 2 3 4 5 6 7 8 9 . , : ; - [ ]

Would it be possible to allow a list of chars to be converted and skip all others, I realise this will need some trickery with the map, but it would dramatically reduce the size for those large fonts where 90% of the chars will never be used.

[peterhinch]

I'll give it some thought, but it may be a while before I get a chance to consider this. Note that if you freeze the font file as bytecode the unwanted glyphs occupy Flash: they won't consume scarce RAM.

[masayoshi-louis]

need this feature too

[peterhinch]

Could you guys please explain the use case for this? If you freeze a font as bytecode, its import only consumes about 300 bytes of RAM. This would not be reduced by the proposed change.

[masayoshi-louis]

I am using d1 mini, which has 4mb flash.

Currently, I freeze a font file and some modules. The size is:

esptool.py v1.2
('flash    ', 35520)
('padding  ', 1344)
('irom0text', 576708)
('total    ', 613572)

After adding two more font files (courier20.py and freesans20.py), my build fails:

LINK build/firmware.elf
xtensa-lx106-elf-ld: build/firmware.elf section `.irom0.text' will not fit in region `irom0_0_seg'
xtensa-lx106-elf-ld: region `irom0_0_seg' overflowed by 360 bytes
Makefile:187: recipe for target 'build/firmware.elf' failed
make: *** [build/firmware.elf] Error 1

Now I'm trying to adjust the size of "irom0_0_seg". But it will be great if we can save some space.

[peterhinch]

@masayoshi-louis Thanks for the clarification. I have pushed a version which implements this.

[masayoshi-louis]

Got an error

./micropython-font-to-py/font_to_py.py RobotoMono-Regular.ttf 18 robotomono18.py -c 1234567890: Writing Python font file. Traceback (most recent call last): File "./micropython-font-to-py/font_to_py.py", line 552, in args.errchar, args.charset): File "./micropython-font-to-py/font_to_py.py", line 389, in write_font fnt = Font(font_path, height, minchar, maxchar, monospaced, defchar, charset) File "./micropython-font-to-py/font_to_py.py", line 274, in init self.max_width = self.get_dimensions(size) File "./micropython-font-to-py/font_to_py.py", line 293, in get_dimensions glyph = self._glyph_for_character(char) File "./micropython-font-to-py/font_to_py.py", line 315, in _glyph_for_character freetype.FT_LOAD_TARGET_MONO) File "/home/luyi/.local/lib/python3.6/site-packages/freetype/init.py", line 1308, in load_char char = ord(char.decode('utf8')) AttributeError: 'str' object has no attribute 'decode'

[peterhinch]

I haven't got that ttf, but I downloaded roboto.otf and failed to replicate the problem: it produced the Python file which rendered correctly. The traceback makes me wonder if there is a freetype version issue. At a Python 3.x prompt enter

import freetype
freetype.version()

I get (2, 5, 2). The point in my code where the error occurs is not part of the code which has changed. Did you have success with the previous version of font_to_py.py?

If upgrading freetype doesn't fix the problem perhaps you could point me to a source for the font file (or send it to me) so I can exactly replicate your command line.

[masayoshi-louis]

my freetype version is (2, 9, 1)

The current version (master branch) works without '-c' parameter.

And the same problem can also be reproduced using FreeSansBold.ttf.

[peterhinch]

I'm baffled:

[adminpete@axolotl]: /mnt/qnap2/data/Projects/MicroPython/micropython-font-to-py
$ font_to_py.py FreeSansBold.ttf 18 freesansbold.py -c 1234567890:
Writing Python font file.
Height set in 2 passes. Actual height 18 pixels.
Max character width 15 pixels.
freesansbold.py written successfully.

[peterhinch]

I suggest you run as root

# pip3 install --upgrade freetype-py

When I tried it just now it installs version (2, 5, 2). I know your version sounds more recent, but I'm not sure how that's possible: is it perhaps a fork?

[masayoshi-louis]

I think the problem is caused by different version of freetype.

pip3 provides 0.1, 0.1.1, 0.2.0, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.4, 0.4.1, 0.4.2, 0.5.0, 0.5.1, 0.5.3, 1.0, 1.0.1, 1.0.2, 1.1, 1.2, 1.2.1, 2.0, 2.0.0.post1, 2.0.0.post2, 2.0.0.post3, 2.0.0.post4, 2.0.0.post5, 2.0.0.post6

My version is 2.0.0.post6 (default).

[peterhinch]

Have you tried

# pip3 install --upgrade freetype-py

When I tried it, it uninstalled the existing version and installed (2, 5, 2). The application worked.

I'm no expert on PyPi, so I'm at a bit of a loss here. Installing as per the instructions on the PyPi freetype-py page, except for using pip3 for Python 3, produces a version which reports (2, 5, 2) and which enables my code to run.

1. How did you install to get 2.0.0.post6?
2. How do you know you have that version?
3. How can I relate version numbers like 2.0.0.post6 with those reported by the version() method?

The release history shows six "post" versions in quick succession on 3rd July. Prior to that is V2.0 on 27th June. Does the default installation produce V2.0, and the 3rd July releases are buggy?

For what it's worth the part of the code in font_to_py.py which accesses freetype is unchanged in the last 18 months. This takes us back to V1.0.2. The application survived the change to V2. If the latest freetype version fails it suggests that the freetype interface has changed either by a bug or by an API change. I wouldn't expect to see a deliberate API change between versions with such very close numbers.

[masayoshi-louis]

I started a clean docker container:

docker run --name=freetype-py -it ubuntu:16.04 bash

Then in the container:

apt-get update
apt-get install -y python3-pip
pip3 install freetype-py

It prints:

Collecting freetype-py
  Downloading https://files.pythonhosted.org/packages/80/15/e284bb2d3e6a3d42d5455d189084bbff0fa024c2c2f4f0dec66e7667c476/freetype_py-2.0.0.post6-py2.py3-none-manylinux1_x86_64.whl (703kB)
    100% |################################| 706kB 788kB/s
Installing collected packages: freetype-py
Successfully installed freetype-py

From the download address, we know that the version is 2.0.0.post6.

>>> import freetype
>>> freetype.version()
(2, 9, 1)

Finally I make the code working, by:

apt-get install -y libfreetype6
pip3 install freetype-py==1.0.2

[masayoshi-louis]

I found a new use case of this feature.

We can now print CJK chars!

font_to_py.py ms_yahei.ttf 40 ms_yahei40.py -x -c 你好世界

Although the resulting file is massive.

[peterhinch]

We can now print CJK chars!

Impressive!

I can now replicate your result in a fresh VM. I'm still unclear why my Linux box keeps getting V (2, 5, 2). However I now have a VM with (2, 9, 1) so I can make progress.

There is an interface change in FreeType. I'll post an update to font_to_py.py and a comment here when I have a fix which works with both versions.

Thank you for pointing out this issue and your help in isolating it.

[peterhinch]

I've pushed an update which works with both versions.