Create .def and .dat files for packages

[naveen521kk]

Currently, we just copy the default language.dat and language.def and it creates an error because of missing files and asking over at mailing list, it looks like we should create it like how we do for *.fmts and *.maps. See https://tug.org/pipermail/tldistro/2021q2/000415.html.

It's about parsing texlive.tlpdb, the execute AddHyphen sections from them. For example, in a package called hyphen-indic this is the section

execute AddHyphen   name=assamese   lefthyphenmin=1     righthyphenmin=1    file=loadhyph-as.tex    file_patterns=hyph-as.pat.txt   file_exceptions=
execute AddHyphen   name=bengali    lefthyphenmin=1     righthyphenmin=1    file=loadhyph-bn.tex    file_patterns=hyph-bn.pat.txt   file_exceptions=
execute AddHyphen   name=gujarati   lefthyphenmin=1     righthyphenmin=1    file=loadhyph-gu.tex    file_patterns=hyph-gu.pat.txt   file_exceptions=
execute AddHyphen   name=hindi  lefthyphenmin=1     righthyphenmin=1    file=loadhyph-hi.tex    file_patterns=hyph-hi.pat.txt   file_exceptions=
execute AddHyphen   name=kannada    lefthyphenmin=1     righthyphenmin=1    file=loadhyph-kn.tex    file_patterns=hyph-kn.pat.txt   file_exceptions=
execute AddHyphen   name=malayalam  lefthyphenmin=1     righthyphenmin=1    file=loadhyph-ml.tex    file_patterns=hyph-ml.pat.txt   file_exceptions=
execute AddHyphen   name=marathi    lefthyphenmin=1     righthyphenmin=1    file=loadhyph-mr.tex    file_patterns=hyph-mr.pat.txt   file_exceptions=
execute AddHyphen   name=oriya  lefthyphenmin=1     righthyphenmin=1    file=loadhyph-or.tex    file_patterns=hyph-or.pat.txt   file_exceptions=
execute AddHyphen   name=pali   lefthyphenmin=1     righthyphenmin=2    file=loadhyph-pi.tex    file_patterns=hyph-pi.pat.txt   file_exceptions=
execute AddHyphen   name=panjabi    lefthyphenmin=1     righthyphenmin=1    file=loadhyph-pa.tex    file_patterns=hyph-pa.pat.txt   file_exceptions=
execute AddHyphen   name=tamil  lefthyphenmin=1     righthyphenmin=1    file=loadhyph-ta.tex    file_patterns=hyph-ta.pat.txt   file_exceptions=
execute AddHyphen   name=telugu     lefthyphenmin=1     righthyphenmin=1    file=loadhyph-te.tex    file_patterns=hyph-te.pat.txt   file_exceptions=

the corresponding language.def should be

% from hyphen-indic:
\addlanguage{assamese}{loadhyph-as.tex}{}{1}{1}
\addlanguage{bengali}{loadhyph-bn.tex}{}{1}{1}
\addlanguage{gujarati}{loadhyph-gu.tex}{}{1}{1}
\addlanguage{hindi}{loadhyph-hi.tex}{}{1}{1}
\addlanguage{kannada}{loadhyph-kn.tex}{}{1}{1}
\addlanguage{malayalam}{loadhyph-ml.tex}{}{1}{1}
\addlanguage{marathi}{loadhyph-mr.tex}{}{1}{1}
\addlanguage{oriya}{loadhyph-or.tex}{}{1}{1}
\addlanguage{pali}{loadhyph-pi.tex}{}{1}{2}
\addlanguage{panjabi}{loadhyph-pa.tex}{}{1}{1}
\addlanguage{tamil}{loadhyph-ta.tex}{}{1}{1}
\addlanguage{telugu}{loadhyph-te.tex}{}{1}{1}

and for language.dat is should be

% from hyphen-indic:
assamese loadhyph-as.tex
bengali loadhyph-bn.tex
gujarati loadhyph-gu.tex
hindi loadhyph-hi.tex
kannada loadhyph-kn.tex
malayalam loadhyph-ml.tex
marathi loadhyph-mr.tex
oriya loadhyph-or.tex
pali loadhyph-pi.tex
panjabi loadhyph-pa.tex
tamil loadhyph-ta.tex
telugu loadhyph-te.tex

After parsing the file should be saved as <package-name>.def and <package-name>.dat, and the hook needs to be updated so that these are added saved as language.* and loaded accordingly.

[naveen521kk]

Further looking, the format of language.def should be according to the comment below.

% The five parameters for \addlanguage are:
% language, patterns file, exceptions file, left- and right-hyphen minima

For language.dat it should be

<name> <file>
= <synonyms-1>
= <synonyms-2>
...

[naveen521kk]

There's language.dat.lua need to check what should be done there. it's something like for the same hyphen-indic

-- from hyphen-indic:
    ['assamese'] = {
        loader = 'loadhyph-as.tex',
        lefthyphenmin = 1,
        righthyphenmin = 1,
        synonyms = {  },
        patterns = 'hyph-as.pat.txt',
        hyphenation = '',
    },
    ['bengali'] = {
        loader = 'loadhyph-bn.tex',
        lefthyphenmin = 1,
        righthyphenmin = 1,
        synonyms = {  },
        patterns = 'hyph-bn.pat.txt',
        hyphenation = '',
    },
    ['gujarati'] = {
        loader = 'loadhyph-gu.tex',
        lefthyphenmin = 1,
        righthyphenmin = 1,
        synonyms = {  },
        patterns = 'hyph-gu.pat.txt',
        hyphenation = '',
    },
    ['hindi'] = {
        loader = 'loadhyph-hi.tex',
        lefthyphenmin = 1,
        righthyphenmin = 1,
        synonyms = {  },
        patterns = 'hyph-hi.pat.txt',
        hyphenation = '',
    },
    ['kannada'] = {
        loader = 'loadhyph-kn.tex',
        lefthyphenmin = 1,
        righthyphenmin = 1,
        synonyms = {  },
        patterns = 'hyph-kn.pat.txt',
        hyphenation = '',
    },
    ['malayalam'] = {
        loader = 'loadhyph-ml.tex',
        lefthyphenmin = 1,
        righthyphenmin = 1,
        synonyms = {  },
        patterns = 'hyph-ml.pat.txt',
        hyphenation = '',
    },
    ['marathi'] = {
        loader = 'loadhyph-mr.tex',
        lefthyphenmin = 1,
        righthyphenmin = 1,
        synonyms = {  },
        patterns = 'hyph-mr.pat.txt',
        hyphenation = '',
    },
    ['oriya'] = {
        loader = 'loadhyph-or.tex',
        lefthyphenmin = 1,
        righthyphenmin = 1,
        synonyms = {  },
        patterns = 'hyph-or.pat.txt',
        hyphenation = '',
    },
    ['pali'] = {
        loader = 'loadhyph-pi.tex',
        lefthyphenmin = 1,
        righthyphenmin = 2,
        synonyms = {  },
        patterns = 'hyph-pi.pat.txt',
        hyphenation = '',
    },
    ['panjabi'] = {
        loader = 'loadhyph-pa.tex',
        lefthyphenmin = 1,
        righthyphenmin = 1,
        synonyms = {  },
        patterns = 'hyph-pa.pat.txt',
        hyphenation = '',
    },
    ['tamil'] = {
        loader = 'loadhyph-ta.tex',
        lefthyphenmin = 1,
        righthyphenmin = 1,
        synonyms = {  },
        patterns = 'hyph-ta.pat.txt',
        hyphenation = '',
    },
    ['telugu'] = {
        loader = 'loadhyph-te.tex',
        lefthyphenmin = 1,
        righthyphenmin = 1,
        synonyms = {  },
        patterns = 'hyph-te.pat.txt',
        hyphenation = '',
    },

[naveen521kk]

execute AddHyphen   name=afrikaans  lefthyphenmin=1     righthyphenmin=2    file=loadhyph-af.tex    file_patterns=hyph-af.pat.txt   usenglish=hyph-af.hyp.txt

I found the parameter usenglish to be a bit weird and unexpected. Shouldn't it be file_exceptions instead of usenglish.

I could see that it is in language.dat.lua

-- from hyphen-afrikaans:
    ['afrikaans'] = {
        loader = 'loadhyph-af.tex',
        lefthyphenmin = 1,
        righthyphenmin = 2,
        synonyms = {  },
        patterns = 'hyph-af.pat.txt',
        hyphenation = 'hyph-af.hyp.txt',
    },

wherein other places hyphenation is the file_exceptions parameter in the tlpdb file.

@norbusan can you help?

[norbusan]

That is indeed strange, can you send me an email and/or the TL list to remind me, in currently in the mountains

[naveen521kk]

Oh, I was using an old texlive.tlpdb for a reference locally, getting the latest one that was fixed. Anyway, I will send you an email with the file I have. Also, have a nice time!

[naveen521kk]

This is fixed in #5