Missing E_TEHTA_GRAVE_INF (and other possible 'below' tehtar)

[laicasaane]

There are many tehtar that don't have a below counterpart, should we add them to the fonts?

They aren't defined in the standard unicode layout and the Free Tengwar Font, and are somewhat specific (for now) just for my Vietnamese mode, so my first thought is I should add them manually for my own, but I don't know if there is a toolchain to build the sdfs? When I generated Tengwar Annatar GlaemUnicode for testing, I see a big different in size (mine is 70KB, yours is 97KB).

So if you don't want to add these specific tehtar to the standard fonts here, please give me some guidelines to build the sdfs. Thank you. :)

[BenTalagan]

There are many tehtar that don't have a below counterpart, should we add them to the fonts?

Well, the short answer is no. Theoretically you should not have relied on something which is not attested (and the norm is by will really strict on that matter). But in the meantime, to my opinion, extrapolating strict 'below' counterparts of 'upper' tehtar is not the craziest idea. It may be worth checking the latest Parma Eldalamberon if these versions can't be found, which could (maybe) help legitimate the fact that the norm is missing some stuff (which I'm already convinced, on the specific matter of sa-rinci for example).

They aren't defined in the standard unicode layout and the Free Tengwar Font, and are somewhat specific (for now) just for my Vietnamese mode, so my first thought is I should add them manually for my own, but I don't know if there is a toolchain to build the sdfs?

Ok so what I don't understand is : were you using these tehtar before ? They were not present in my legacy *_ds_* charsets so I'm just curious if it's a new problem, or something you were able to workaround before thanks to your own charsets? I think these glyphs are not even present in the legacy fonts themselves! What tehtar are you missing? Anyway, this might not be a good idea to get away from the norm - your mode would definitely lose compatibility with MonoTengwar and Telcontar (which is imho still too much experimental to be fully used yet in its opentype version) and you would be completely stuck with these hacky glaemscribe fonts. What is the list of missing tengwar?

The toolchain is, for the moment, the following :

1. New sfds are generated from the legacy sfds using the commands given in : /fonts/remap_tool/_readme.md . They first generate partial macro files with a list of commands that perform the remapping, by comparing the old charsets to a new, common, target layout (finallayout.cst). They also do other operations on the sfds (like renaming them, duplicating a few missing chars, and removing a few dirty chars). At the moment, new GlaemUnicode sfd files are exactly the result of this step. This means that *_ds_* and *_guni\* charsets are still compatible, but at one stage somewhere in the future, that compatibility may be dropped (as soon as we edit directly the GlaemUnicode fonts, we lose the compatibility). So, to my opinion, adding missing chars should rather be done in the legacy fonts at the moment, to avoid breaking that retro-compatibility.

2. I open these generated sfds with FontForge (they are located in /fonts/src/sfds), and generate TrueType counterparts in /fonts/src/ttf_exports/*-noautohint (you can ignore FF warnings).

3. I generate autohinted versions of these fonts, with the /fonts/src/build.rb script (which calls ttfautohint which does a great job). This is really important if you want a better rendering in some web browsers or rendering engines.

4. I generate web versions of these fonts (which are located in /fonts/build/ttfs) using font squirrel generator.

I think we should really discuss your options before doing anything since you're playing with fire :)

[laicasaane]

What tehtar are you missing?

Currently I'm using THINNAS to represent the "huyền" tone, but visually it's a little bit off, and in some cases would be easily confused with the "sắc" tone (which is represented by E_TEHTA_INF). So my mode is in need of a reversed E_TEHTA_INF to make these 2 tones separated at best.

Anyway, this might not be a good idea to get away from the norm - your mode would definitely lose compatibility with MonoTengwar and Telcontar (which is imho still too much experimental to be fully used yet in its opentype version) and you would be completely stuck with these hacky glaemscribe fonts.

You know, before there is no Unicode for Vietnamese Latin alphabets, our alphabets were also lost compatibility with "standard" English fonts too. Even nowaday, not many Unicode fonts support all Vietnamese charset. I fully understand this risk of adding custom characters to the norm. So let's say, if someone somewhere decides to use my Vietnamese mode, they would need to install a Vietnamese supported font. (And they must accept that not all fonts would be compatible with Vietnamese mode.) It's inevitable.

[BenTalagan]

Ok, I understand perfectly your needs and your point of view.

The strictness of the norm is somewhat discussable on the specific point of down tehtar counterparts. For example, in the latest PE on pages 11 and 14, we have something called watehta, which is an upper circle tehta, and a variant of the labial mark. Maybe it was dropped by Tolkien because it's hardly combinable with something else. This tehta is not in the norm but probably could be (or even, should be). It is hard to guess what could exist in other unpublished texts, but extrapolating 'below' counterparts is not an absolute sacrilege to my opinion, if we see it as the completion of a general process used by Tolkien. So in the absolute, they are not attested, but the principle leading to them is. One could even argue that the norm should contain the same number of upper and below tehtar - but it's a bit more discussable, especially for "big" tehtar which do not fit really well under tengwar.

Anyway, legacy fonts also contain neo-tengwar characters (such as the 5 dots double a-tehtar, which is imho far more a sacrilege than the reversed below e-tehta :p because it's pure graphical invention), and I have backported them, and they are not compatible with the norm. So I can probably make an extra and add E_TEHTA_GRAVE_INF, especially if it is the only one you're missing :)

I'll probably do it next week-end. There's a bit of work (7 fonts) but it can probably be done quick (copy/paste/mirror).

[laicasaane]

Ah yes, my only need for now is an E_TEHTA_GRAVE_INF. The others were meant to be discussed only. IMO, the tengwar are fairly symmetric, so some tehtar can be both above and below without problem, but until now, the below counterparts are missing. Just because they are unattested it doesn't mean it should be that forever? When people starts adopting Tengwar the need of custom glyphs would eventually arise sooner or later (maybe later, somewhat far off in the future?).

[BenTalagan]

In fact, if we have a close look at the norm, there aren't many counterparts missing (http://freetengwar.sourceforge.net/mapping.html).

The column E04X is completely symmetric.

And in column E05X, I would find logical to have only a counterpart for E054, E056 and maybe E055. Inversely, E07D could have an upper counterpart too.

If we wanted to add these unattested-but-coherent glyphs to the norm, the cleanest way would probably be to use a separate block of the PUA. But for now, the problem does not even exist (in the glaemunicode transitional fonts the tehtar slots are blank because they exist at another place in multiple versions).

Well, if it's ok for you, I will only add E_TEHTA_GRAVE_INF, and keep in mind this matter.

[BenTalagan]

Ok, done in the unicode_font_remapping branch. It was a bit of a pain since there's more and more things to do to manage backward compatibility. The result, as for all 'below' tehtar, is not exceptional (with tengwar such as lambe it's really awful). I ponder the fact of designing my own fresh OpenType tengwar font with guidelines to achieve all combinations instead of spending time to port things to these old fonts (but still that's a pity since some of them are really beautiful).

[BenTalagan]

One more thing, I have renamed the current development branch to NextRelease.1.2.0 (FYI) :-) .

[laicasaane]

I ponder the fact of designing my own fresh OpenType tengwar font

Only if you have time to work on it. Fixing existing fonts still requires less time and effort. Maybe you should list that OpenType font as a long time project?

[BenTalagan]

Yes. I had a closer look at how Telcontar was made by Johan and Mach and it's quite some work. But the technical approach (starting from metafont and coding the whole font behaviour) is truly awesome. And it proves how hard it is for a designer alone to make such a font, since there's a lot of coding complexity around the corner (the drawing part is great and the opentype part is really a complex headache).

You're right it's a long time project, there are prior things to do on the modes atm. Finishing the english mode is one of them ; it's a mid-term project, the technical principles are all validated, but we (Bertrand and I) are forking the IPA generation from espeak-ng to reorder how things are done and to add phonetical details with options. A short-term project is to publish 1.2.0 which is almost ready ; the only things left is that we are enhancing the sindarin modes atm (to add options and fix very precise phonological problems).

[laicasaane]

English mode! Wow! I'm still transcribing it manually. Looking forward to see it go live.

[laicasaane]

Hi, today I've played around a bit with the E_TEHTA_GRAVE_INF and I found out that, in some fonts, its position looks imprecise. Here is the comparision between the actual (left) and the expected position (right) that should be:

[BenTalagan]

Hi Laicassane, thanks for your feedback.

I quite agree with you. I think I used a mirrored version of the normal inferior E tehta, and that made these variants a bit too much on the left because they were mirrored on the middle axis of the tehta as a reference, not the top tip. You can have a look at the comparison charts in font/virtual_chars_tool, (it's a tool that I wrote and use to make choices between variants) and you will see more clearly how these variants appear. For each font I had already selected the variant which works best, but it's not sufficient.

However, it may prove difficult to succeed in having four, well placed, functional variants of this tehta. It's really large... I'll have a try in fixing this but it may take some time (as you could see, I haven't yet released 1.2.0 due to the fact that it was hard to synchronize this year with my mate Bertrand on the things we wanted to release for the website).

Anyway, I've started working on a draft for new fonts. It's going to be a long running and difficult project imho, but probably worth the work when we see that dealing with legacy fonts is such a painful task.

[laicasaane]

I'll have a try in fixing this but it may take some time

You could take all the time you need. There is no pushing from my part. I'm grateful for your help working on these unusual requests of me, really.

Anyway, I've started working on a draft for new fonts.

Could you elaborate in another issue? I'd like to know your idea and plan about it.

And, FYI:

Laicassane

It's "saane" (Quenya: sáne). For my real name means "green pine" literally, thus "laica sáne" in Quenya. 😄

[BenTalagan]

It's "saane" (Quenya: sáne). For my real name means "green pine" literally, thus "laica sáne" in Quenya. 😄

Oops sorry. I know it but was a bit tired yesterday :)

You could take all the time you need. There is no pushing from my part. I'm grateful for your help working on these unusual requests of me, really.

Thanks :-)

Anyway, I've started working on a draft for new fonts.

Could you elaborate in another issue? I'd like to know your idea and plan about it.

Actually it's more like a brand new project than a new issue ; so I might stay synthetic here. At first I had been reviewing the code for Telcontar to (possibly) fork it. I'm not a fan of how the small sa-rince is rendered in Telcontar (below the tengwa, instead of at the right end of the tengwa), and there are other small things that I would have liked to tweak (the new UI of Glaemscribe on Glaemscrafu will be proposing the use of Telcontar and I've noticed several bugs). But the dependency to metafont is really hard to deal with, there's only a few documentation on metafont, and I felt like I had hit a wall here, things don't feel natural at all for me with metafont. But the general flow designed by Johann and Mach is great. So I want to build some kind of framework and flow inspired by Telcontar, but written in a more modern language (like JS) that would enable to perform procedural calligraphy in a more "photoshop like" coded way (with pen shape, pressure, deformation, rotation, etc). It would allow the same benefits as metafont : draw the vast majority of tengwar in a row, change their tilt, weight, etc with just a few parameters without redrawing everything by hand and have the insurance that everything is coherent. But, in addition, opentype instrumentation (placing anchors e.g) could be done at the same time, it would be really easy to do since it's part of the shape of a character. As well, the vectorization would be done by potrace. Auto-kerning algorithms would be quite easy to write since, at the basis, characters are pre-rendered. But anyway, it's just ideas for now, and the stage is pretty early. I've only started niggling with it a few days ago, so nothing to show here :)

[laicasaane]

Well, it seems to be a huge project. But I cannot say that I understand all of what you've said. However, it does feel interesting (except the JS part - I'm always lost reading dynamic languages 😢). Do you think I might be able to help to some extent?

[TroyAlford]

I'd be interested in potentially contributing as well. I know very little about fonts in general, but I write and teach JavaScript for a living - and I'd love to build a better integrated version of react-glaemscribe than what I have right now.

[BenTalagan]

Thanks a lot to both of you! I'd definitely be delighted to share some thoughts when (and if) the project gets stronger. Trying to build a first prototype of a rendering engine and experimenting with it for the last few days tends to show me things can be quite tricky - dealing with a cool model, pixel precision, and performance is quite a headache :)

[BenTalagan]

Hi Laicasaane, back to the original issue : I've finally taken the time to fix this tehta for all tengwar fonts (except elfica, where it was ok). You can checkout them from the NextRelease.1.2.0 branch. Hope this time it will do the job!

[BenTalagan]

Closed after 1.2.0 release.