Unicode support

[virxkane]

Hello AdafruitGFX members! Not so long ago I wrote my own library for rendering bitmap fonts, created by the fontconvert utility from this project. I also modified this utility. This modified utility, unlike the original, allows you to save glyphs in a bitmap font not from one range, but from several. Moreover, when writing character codes, a 32-bit integer is used, i.e. any unicode characters can be included in the generated font. Due to the ability to include characters from several ranges in one font file, characters from several languages can be embedded in it, eliminating unnecessary gaps. This avoids creating multiple files of the same font for each language and switching between them.
Of course:

• this will increase the bitmap font size by a few bytes, but it can be reduced slightly by eliminating unused characters in between;
• this will slightly increase the binary code of the library, but such is the price of unicode;
• this required changing the gfxfont.h file.

It also uses font hinting to make the outline of the font appear more uniform.

If you are interested in adding Unicode support to this project, sources of modified fontconvert utility: https://github.com/virxkane/fontconvert
Sources that render such fonts: https://github.com/virxkane/nucleo-f446re-st7735spi-shield
Most interested code: https://github.com/virxkane/nucleo-f446re-st7735spi-shield/blob/main/Src/gfx.c
All my code is written under BSD-3 license, hopefully it is compatible with this project's license.

I have not studied and never used Adafruit-GFX-Library, I do not know the structure of this project, so I cannot make a PR, but I will be glad if someone does it using my code.

[alejandro13041]

Can you specify more of this please, how i can use your master tool?

[virxkane]

@alejandro13041

Can you specify more of this please, how i can use your master tool?

How to use modified fontconvert:

• compile using cmake
• generate some fonts definitions using this utility, for example, using mk_sample.sh script or with direct call:

  fontconvert --size=7 --ascii --dpi=116 --hinting=auto /usr/share/fonts/noto/NotoSans-Regular.ttf > NotoSans-Regular-7pt-ascii.h
  fontconvert --size=7 --chars=0x20-0x7E,0x410-0x44F,0x401,0x451,0xB0 --dpi=116 --hinting=auto /usr/share/fonts/noto/NotoSans-Bold.ttf > NotoSans-Bold-7pt-ascii+degree+rus.h

Usage:

Usage: fontconvert <options> [font_file]
font_file - path to the font file.
options:
--size=<font_size>           |-s        specify font size
--chars=<f1-l1,f2-l2,cc,...> |-r        specify characters set as range list:
                                        where f1 - first char codepoint in range 1;
                                        where l1 - last char codepoint in range 1;
                                        where f2 - first char codepoint in range 2;
                                        where l2 - last char codepoint in range 2;
                                        where cc - one char codepoint to add to range;
                                        etc...
--onechar=<code>             |-c        specify only one character
--ascii                      |-a        specify ACSII mode: one charset range: first=0x20, last=0x7E
--dpi=<dpi_value>            |-d        specify DPI
--hinting=[no|mono|auto]     |-t        specify hinting mode
--progmem[=1|0|yes|no]       |-d        use 'PROGMEM' specification for font data declarations
--help                       |-h        show this page and exit.

Then you can use this font in your projects, but resulting font files isn't compatible with Adafruit-GFX-Library! You should use your own font drawing routine, for example, like this: https://github.com/virxkane/nucleo-f446re-st7735spi-shield/blob/main/Src/gfx.c

[alejandro13041]

@alejandro13041

¿Puedes especificar más sobre esto por favor? ¿Cómo puedo utilizar tu herramienta maestra?

Cómo utilizar fontconvert modificado:

• Compilar usando cmake
• generar algunas definiciones de fuentes usando esta utilidad, por ejemplo, usando mk_sample.shun script o con una llamada directa:

fontconvert --size=7 --ascii --dpi=116 --hinting=auto /usr/share/fonts/noto/NotoSans-Regular.ttf > NotoSans-Regular-7pt-ascii.h
fontconvert --size=7 --chars=0x20-0x7E,0x410-0x44F,0x401,0x451,0xB0 --dpi=116 --hinting=auto /usr/share/fonts/noto/NotoSans-Bold.ttf > NotoSans-Bold-7pt-ascii+degree+rus.h

Uso:

Usage: fontconvert <options> [font_file]
font_file - path to the font file.
options:
--size=<font_size>           |-s        specify font size
--chars=<f1-l1,f2-l2,cc,...> |-r        specify characters set as range list:
                                        where f1 - first char codepoint in range 1;
                                        where l1 - last char codepoint in range 1;
                                        where f2 - first char codepoint in range 2;
                                        where l2 - last char codepoint in range 2;
                                        where cc - one char codepoint to add to range;
                                        etc...
--onechar=<code>             |-c        specify only one character
--ascii                      |-a        specify ACSII mode: one charset range: first=0x20, last=0x7E
--dpi=<dpi_value>            |-d        specify DPI
--hinting=[no|mono|auto]     |-t        specify hinting mode
--progmem[=1|0|yes|no]       |-d        use 'PROGMEM' specification for font data declarations
--help                       |-h        show this page and exit.

Luego puedes usar esta fuente en tus proyectos, pero los archivos de fuentes resultantes no son compatibles con Adafruit-GFX-Library. Debes usar tu propia rutina de dibujo de fuentes, por ejemplo, como esta: https://github.com/virxkane/nucleo-f446re-st7735spi-shield/blob/main/Src/gfx.c

Could you explain more clearly that your font converter is not compatible with Adafruit-GFX-Library ? I'm trying to convert the NSimSum font to be able to use it with an SSD1306 display but when checking the Adafruit documentation it said that it did not have support for handling UNICODE characters and only handles ASCII, so I thought about modifying the tool for that, but I figured that on the internet someone must have already done it so I found your repository but according to the answer you gave me it says that it does not have support for Adafruit-GFX-Library which is the library that I am using, and that I should use a font drawing routine but I'm not sure how to do that, so then what is the project for? and do you know what possible solution I could give to this?

[virxkane]

Could you explain more clearly that your font converter is not compatible with Adafruit-GFX-Library ?

Because it generates data structures incompatible with Adafruit-GFX-Library. A font is an instance of the GFXFont structure, in my project this structure is changed and incompatible with the original.

I'm trying to convert the NSimSum font to be able to use it with an SSD1306 display but when checking the Adafruit documentation it said that it did not have support for handling UNICODE characters and only handles ASCII, so I thought about modifying the tool for that

You can't just modify the fontconvert utility to support unicode, you need to modify the entire stack of functions that work with fonts.

so I found your repository but according to the answer you gave me it says that it does not have support for Adafruit-GFX-Library which is the library that I am using, and that I should use a font drawing routine but I'm not sure how to do that, so then what is the project for?

I don't offer ready-made solutions for Adafruit-GFX-Library, which I clearly wrote in the very first message. If you are not ready to write low-level functions to working with graphics and text, unfortunately, this will not help you. My project is just a demonstration of how to output text in Unicode on embedded systems, if anyone wants to implement Unicode support in the Adafruit-GFX-Library, then perhaps this will help.

[alejandro13041]

¿Podrías explicar más claramente que tu convertidor de fuentes no es compatible con Adafruit-GFX-Library?

Porque genera estructuras de datos incompatibles con Adafruit-GFX-Library. Una fuente es una instancia de la estructura GFXFont, en mi proyecto esta estructura está modificada y es incompatible con la original.

Estoy tratando de convertir la fuente NSimSum para poder usarla con una pantalla SSD1306 pero al revisar la documentación de Adafruit decía que no tenía soporte para manejar caracteres UNICODE y solo maneja ASCII, así que pensé en modificar la herramienta para eso.

No puedes simplemente modificar la fontconvertutilidad para que admita Unicode, necesitas modificar toda la pila de funciones que trabajan con fuentes.

Entonces encontré su repositorio, pero según la respuesta que me dio, dice que no tiene soporte para Adafruit-GFX-Library, que es la biblioteca que estoy usando, y que debería usar una rutina de dibujo de fuentes, pero no estoy seguro de cómo hacer eso, entonces, ¿para qué es el proyecto?

No ofrezco soluciones listas para usar para Adafruit-GFX-Library, como escribí claramente en el primer mensaje. Si no está listo para escribir funciones de bajo nivel para trabajar con gráficos y texto, lamentablemente, esto no lo ayudará. Mi proyecto es solo una demostración de cómo generar texto en Unicode en sistemas integrados. Si alguien desea implementar soporte Unicode en Adafruit-GFX-Library, entonces tal vez esto le ayude.

Thanks for clarifying, so basically what I have to do is edit the stack of functions that render the screen, I saw that you have a project that prints text in Cyrillic, is that basically what I need?

[virxkane]

I saw that you have a project that prints text in Cyrillic, is that basically what I need?

Yes, there you can see how to display Unicode text on the screen. And then you have to decide what to do with it. Either modify the Adafruit-GFX-Library library to support Unicode or write your own :)

[alejandro13041]

i think that is better modify preexistent library, thanks :)