Complete TODO items and implement UI fixes

[gezakerecsenyi]

This fork is quite messy: all of my commits had to be rebased into one as I was hitting GitHub's 100MB file limit with outputs/random_generated_syllables_with_variations.json and had to convince it that the file never existed... I would highly suggest re-generating everything yourself to prevent any problems arising from the uploading of generated files to GitHub.

I'll do my best to explain my changes in this PR instead:

• Refactoring: typings have been extracted out to a separate types.ts for consistency. Any instances of strings representing IPA are instead given the IPA type, which is synonymous to string, but glosses differently in an IDE, which I found to greatly reduce confusion as someone starting to work on the codebase anew. A few small variable name inconsistencies and spelling mistakes have additionally been corrected. package.json now contains scripts to run the syllable and word generation.
• Parameter extraction: parameters such as filenames, IPA translation specifications, magic constants, ending tables, etc.., have all been extracted to parameters.ts. This makes experimenting with options much easier.
• Algorithmic improvements: this mostly consists of implementing TODO items.
  • getRandomSyllableFromPalette has been improved to optionally palette-cleansing to prevent character reuse (similarly to getRandomSyllable, though a little more crude to avoid massive performance hits), as well as order enforcement. These two options are both attempted in main.ts before resorting to default (i.e. prior) getRandomSyllableFromPalette behaviour.
  • scoreForRandomSyllable has also been improved to provide bonusses for phonemes being roughly corresponding positions and ordering in the original word, as well as providing less severe punishment for substitution of "similar" phonemes (e.g., best -> pest will be graded higher than best -> mest). This is defined using a lookup table in parameters.ts and is primarily based on voicedness pairs or other taxonomic similarities (e.g. nasality of m and n), though (especially for vowels) may be somewhat subjective - as such, I've attempted to keep to the heuristic of using North-American pronunciation as an indicator.
  • Rhoticised vowels substitute their composed forms - many functions don't properly handle phoneme digraphs, and as such reducing these as much as possible (e.g. substituting rhoticised IPA symbols for combined vowel-rhotic graphs) can greatly reduce issues in these cases.
  • findVariants has been greatly reworked and replaced with findEnglishVariants. In terms of identifying endings, it seems to be far more efficient to test the raw English than the IPA, and, since this is information that we have at all points that findVariants is called, this should be okay, particularly when combined with more sensitive detection formulae (e.g. testing create for ^ing yields creating instead of *createing). As such, I've also revamped the specification system for variants, making it easier to add in a whole host of new ones, including comparatives, superlatives, adverbs, and, through some syntactic specification, re*, dis*, un*, pre*, post*, in*, and non* all can act as prefix matchers.
  • Variant generation has been altered so as not to be restricted to searching in syllable codas. This was a nice touch, but it ultimately decimated the usefulness of the feature, as we very quickly burned through usable codas this way. This PR instead allows not only for insertion anywhere, but also replacement, meaning that it is now feasible to use vowels as alternants to form a (in my opinion) fairly aesthetic ablaut-gradation type morphology to words.
• UI optimisations: hyphentication handling has been fixed, as has handling of capital letters. Fully-capitalised English words have fully-capitalised Glish alternatives, fully-lowercase English words remain lowercase in Glish, and any combination will result in first letter capitalisation in Glish form (e.g. testing -> tengst, Testing -> Tengst, testinG -> Tengst, TESTING -> TENGST). Numbers are left as-is if a word is solely digits, or treated as part of a word if combined with alphabetic characters - but never entirely deleted, as before. Dodgy spacing when using the "Copy monosyllabic" feature of the UI has also been rectified.

Test cases

Original

Type a sentence or paragraph in here in English to see its translation. This is some additional example text to demonstrate.

Old Glish

Type a stes or preft in here in glish to see its shlayr. This is some shuh plimdz to smud.

New Glish

Type a snuns or erg in here in Glish to see its talsh. this is some myih zul text to drayt.

[paralogical]

What a PR! Thank you so much for your work on this, it's really quite comprehensive. Apologies for my delay in responding to this. I'm very happy anyone would be willing to dig through and update my code so much :)

As you mention it's a lot of changes in one big chunk, so it's a little hard to review, but glancing through things generally look better. I appreciate that you extracted parameters into their own file.

My only concern is that there's something nice about leaving the original glish that closely matches the video. I know it already doesn't quite match exactly in some places, but it might be confusing if all the words are regenerated entirely. Even with all these improvements, I will say the example sentence is still pretty similarly tough to understand, and if the output is suddenly totally different, it's hard to use this tool consistently.

I might need to add some kind of versioning system so that baseline improvements can be made while still being possible to see the "original" Glish.