Reimport of header file looses next-to-last entry

[clanganke]

Hi all, thanx for this great utility! I hope it can help me to add german "umlaut" characters (0xc4, 0xd6, 0xdc, 0xdf, 0xe4, 0xf6, 0xfc) to a given font.

Since this seems to be rather complicated, I intend to backup a new header file after each character added, and reimport it before adding the next character. Unfortunately, when reimporting a previously stored header into the form, the next-to-last character gets lost, and the previously added character is displayed at its position, being marked as inprintable.

That means that after having reimported a font with e.g. the 0xc4, the entry for 0xc3, which should be empty, shows the omage of the added 0xc4 character (yet still being marked as inprintable!), and the entry for the 0xc4 character is lost.

Beside the problem, I would like to ask if it a copy-character function could be added - this would come in very handy for special charcters requiring only some pixels being added to an existing character and could help for many (not only german) umlauts.

Thanx a lot in advance!

[clanganke]

I just realized that the problem does not appear with all fonts. I had the problem with a font having been created http://oleddisplay.squix.ch/#/home, however it does not appear with a font coming with the TFT_eSPI lib.

Each I imported once into the customizer, first exported it unchanged, then added the 0xc4 chacacter and then exported it again.

Please see the attached headers testfonts.zip, importing DejaVueSans12\002_exported_c4_added.h into the customizer shows the described problem, whereas importing FreeSans12pt7b\002_exported_c4_added.h shows the proper 0xc4 character at the correct place.

[clanganke]

Sorry to say that, but another porblem shows up. With the font FreeSans12pt7b mentioned in the last comment, where I was able to add one chacater, for each character I add, the previous one shows up all pixels white. So to say

import original - add 0xc4 image - export import - 0xc4 image is ok - add 0xd6 image -export import - 0xc4 image is all white - 0xd6 image is ok - add 0xdc image - export import - 0xc4 and 0xd6 are all white - 0xdc image ist ok ... and so forth...

Looking at the generated data and comparing the result of the different steps, it seems as for the new characters both the bitmap data is being messed with as well as the offsets into the bitmap data go wild. Can you reproduce the problem?

[tchapi]

Hi @clanganke

Thanks for your feedback ! I will try to reproduce when I have some time and see if I can do something. I'll keep you posted. On a side note, you shouldn't have to export / reimport for each character as you can add multiple characters in a range and just flag the ones you don't want so they don't appear in the export

[tchapi]

I have tried to reproduce with the DejaVueSans12 font but I don't see a problem — all characters are kept in their respective place, and it's all good.

When looking at 002_exported_c4_added.h I see that you have a rogue line feed (non-printable NEL character) in the middle of the file (line 162) that can be misinterpreted by some browsers I guess. I have made a fix right now to change the comments when exporting. Can you retest your whole process now ?

[clanganke]

Thnx for the quick research and response. I see that you removed the single quotes from the comments of the generated output. I cannot figure out the problematic character in 002_exported_c4_added.h you mention, is it the UTF-8 encoded character in the comment? The character at which offset is it?

Unfortunately both problems persist, and I am surprised that you cannot reproduce the problem.

First of all, as described before, reimporting the exported header file of DejaVueSans12 after having added the 0xc4 character, it is shown at the 0xc3 position, and the 0xc4 position is lost.

Second, adding one character to FreeSans12pt7b works, but adding a second will leave all pixels of the first added character white, and so forth for every character added after this.

I use Firefox 88.0.1 (64-bit Windows 10) here, pointing at https://tchapi.github.io/Adafruit-GFX-Font-Customiser/.

I have no current desktop Linux around here, but only an old Ubuntu 18 VM with Firefox 65, where I could reproduce both problems.

For the second problem I found an interesting detail, please find testfonts2.zip attached , where I added the exported headers for both platforms after 1) initial export 2) having added an 0xc4 character (dummy pixel data!) 3) having added an 0xd6 character (dummy pixel data!)

If ypou look at the *_win.h files the data of the last character of the initial export start at offset 1962 (001_exported_nochange_win.h line 264). After having added the 0xc4 character, the data of it starts at offset 1969 (002_exported_c4_added_win.h line 335). But after having added the 0xd6 character, the data of the 0xc4 character starts at offset 1962 again (003_exported_d6_added_win.hline 335). Please note that the data of the 0xc4 and 0xd6 character may well differ between the _win.h and the _linux.h versions since I just added some dummy data for a quick test only, not being identical in both cases.

BTW it is perfectly clear that I could add multiple characters at once, but if the problem persists, I wouldn't be able to make any changes to an existing export without losing data.

I appreciate your work very much and hope that this does not take too much of your time!

[tchapi]

Ok, understood. I will need a bit more time to dig deeper then

[tchapi]

Could it be linked to #26 ? I have a made a fix to resolve this

[clanganke]

I see the point, however I am sorry to say that both problems persist even if I remove all comments from every exported file. Moreover, both problems showed up before you removed the single quotes from the comments, where #26 seems to be a consequence of that change.

[tchapi]

Ok. Thanks for your help on this. Continuing investigation :)

[tchapi]

Ok, a first thing to note: your DejaVueSans12\original.h (the font which seems to have a problem) seems to be incorrect as the last glyph exported is indicated as 0x7E whereas it's 0x7D (which is }):

// ...
      {   660,   6,  11,   9,    1,   -9 } // '}'  <<----- this is 0x7D
};
const GFXfont DejaVuSans12 PROGMEM = {
(uint8_t  *)DejaVuSans12Bitmaps,(GFXglyph *)DejaVuSans12Glyphs,0x20, 0x7E, 15};

If you look at the other font (FreeSans12pt7b/original.h), that works well, the last glyph exported is correct:

// ...
    {1962, 10, 5, 12, 1, -10}}; // 0x7E '~'

const GFXfont FreeSans12pt7b PROGMEM = {(uint8_t *)FreeSans12pt7bBitmaps,
                                        (GFXglyph *)FreeSans12pt7bGlyphs, 0x20,
                                        0x7E, 29};

// Approx. 2641 bytes

Thus, there might be something off with the http://oleddisplay.squix.ch/#/home tool, or I have misinterpreted what needs to be indicated in the GFXfont struct at the end of the file., maybe ?

Happy to hear your thoughts on that

[clanganke]

Congrats, VERY WELL FOUND!!! I corrected the original header and the first of the two problems (the one describted in the title of this issue!) does not show up anymore. Thanx a lot for your time investigating this!

It turns out that your tool does a better job compared to the squix font converter at least in one, but important aspect - by just adding the hex values to each glyph line as a comment, as your tool does, such errors would be much easier detectable. Will give squix78 a hint, so that he for one can fix and otherwise possibly extend his great tool, and will for sure credit you for your finding!

Would it be out of scope to add a little check to your code and produce a warning if the amount of glyph data and the data in the font structure do not match?

I just retested the second problem with the now fixed DejaVuSans Font, and it showed up with it as well. Would you prefer that I open a 2nd issue for that remaining problem so that this was better distinguishable, or would we better stick to this issue with it?

[tchapi]

Would it be out of scope to add a little check to your code and produce a warning if the amount of glyph data and the data in the font structure do not match?

Good idea, will try to implement that

I just retested the second problem with the now fixed DejaVuSans Font, and it showed up with it as well. Would you prefer that I open a 2nd issue for that remaining problem so that this was better distinguishable, or would we better stick to this issue with it?

I think it will be useful to have a specific issue for that, yes, if you don't mind :) I will find some time in the week I hope

[tchapi]

Ok, don't bother, I think I found the fix with https://github.com/tchapi/Adafruit-GFX-Font-Customiser/commit/1c4263cece99e4208b93d76bcd37f4e82f6dd13c

[clanganke]

Thank you very much for your efforts, that's just great - I was able to add multiple characters!