Reorganise and add preprocessor

simoncozens commented 10 months ago

This PR:

Creates a set of "input" nam files which are run through a human-friendly preprocessor to generate the "output" nam files
Moves these "output" nam files into the Lib/gfsubsets/data directory, to be used by a soon-to-come Python library.
Systematizes the additional codepoints in certain nam files (such as those using syllabic shaping and CJK punctuation).
Regenerates and updates the "output" nam files to match changes in Unicode 14 and 15.

The only codepoints removed from nam files are those which absolutely shouldn't be there. For example, when we had sample texts in transliterated Sanskrit, we tried adding things like DEVANAGARI SIGN AVAGRAHA to various subsets to try to make them work. That wasn't a good idea and they should be removed. Otherwise, everything else is a superset of what was there before.

rsheeter commented 10 months ago

Could we separate restructuring without changing the subset:codepoints mappings from altering what's actually in the subsetes? - the former is fine, the latter we want to be somewhat careful with as it will impact user traffic. It's hard to understand what mappings changed when it's all mixed together.

davelab6 commented 10 months ago

Could we separate restructuring

isn't that in the commit structure already?

rsheeter commented 10 months ago

I mean separate PRs as the natural way to review is per PR not per commit

simoncozens commented 10 months ago

OK, I split this into 2 PRs, #3 which is the do-nothing reorganisation, and then this one which applies on top of it.

rsheeter commented 10 months ago

Ty. If you set the base branch for this then we'll see the right diff by default. GH has become good about updating it on merge (e.g. merge #3, this one will then target main)

garretrieger commented 10 months ago

I merged the other PR, could you rebase this one and then I'll review.

googlefonts / nam-files

Reorganise and add preprocessor #2