Add CLDR character sets

[davelab6]

As per https://github.com/davelab6/pyfontaine/issues/18#issuecomment-33804457 we should import CLDR charsets too.

Thanks to Cibu for this tip:

See http://unicode.org/repos/cldr/trunk/tools/java/org/unicode/cldr/draft/ and GeneratePickerData.java

Data is available at in viewable form: http://unicode.org/repos/cldr/trunk/tools/java/org/unicode/cldr/draft/picker/

All open-source.

[vitalyvolkov]

Checked out the sources of java. A lot of not understandable issues like where CLDR must be got from. I found this repo: http://unicode.org/Public/cldr/25/. This is latest CLDR datas. Is that correct data for CLDR?

[davelab6]

Yes, v25 from http://cldr.unicode.org/index/downloads is just released.

[davelab6]

You could use SVN to get it from http://unicode.org/cldr/trac/browser/tags/release-25 - or that site offers a ZIP at http://unicode.org/cldr/trac/changeset/10042/tags/release-25?old_path=%2F&format=zip

[vitalyvolkov]

Found that data for CLDR. What is characters inside brackets {and }?

a á b c {cs} {ccs} d {dz} {ddz} {dzs} {ddzs} e é f g {gy} {ggy} h i í j k l {ly} {lly} m n {ny} {nny} o ó ö ő p r s {sz} {ssz} t {ty} {tty} u ú ü ű v z {zs} {zzs}

[vitalyvolkov]

./bin/pyfontaine --missing --collections cldr /usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf > ~/Downloads/pyfontaine.log

pyfontaine.log

To work with CLDR you have to install PyICU package. Please provide me what should be changed or improved.

Things I know is

• [x] to replace locale abbreviation by language, e.g. ja -> Jananese
• [x] to fix escaped character, eg }
• [ ] to expand unknown character {ddz} Now I am trying to find out what is that in xml. Pyfontaine compare unicode codes of available glyphs in font. Can font contain double glyph inside like អ and ◌ា in ?

[davelab6]

I should find this out; assigning to me

[cibu]

I believe this explains that they are indexing characters or sequences: http://cldr.unicode.org/development/development-process/design-proposals/index-characters