search

keymanapp / keyman

Keyman cross platform input methods system running on Android, iOS, Linux, macOS, Windows and mobile and desktop web
https://keyman.com/
Other
372 stars 102 forks source link

chore(common): Update standards data #11003

Open ermshiperete opened 2 months ago

ermshiperete commented 2 months ago

@keymanapp-test-bot skip

keymanapp-test-bot[bot] commented 2 months ago

User Test Results

Test specification and instructions

User tests are not required

Test Artifacts

mcdurdin commented 2 months ago

We just have to be a little careful with testing in case metadata formats have changed.

ermshiperete commented 2 months ago

One thing that changed is in iso639-3.tab the Language_Type column now has H (Historical) instead of A (Ancient) for several languages. But I'm not sure that information is used anywhere in our code.

Other than that it's only updated data AFAICT.

rc-swag commented 2 weeks ago

Here are some observations using the Delphi debugger. However, I really don't understand the whole code. There is an error about inserting dulplicates 

class procedure TISO6393ToBCP47Map.Fill(dict: TDictionary<string, string>);
begin
  dict.Add('aar','aa');
  dict.Add('abk','ab');

For the failing test of sqt-YE the following if statement in CanonicalLanguageCodeUtils.FindBestTag evaluates to true and exits. For the master branch this if statement evaluates to false. 

if not TLangTagsMap.LangTags.TryGetValue(t.Tag, LangTag) then
    begin
      // Not a known tag but perhaps it's a custom language
      // We'll make no further assumptions
      Exit(t.Tag);
    end;
rc-swag commented 1 week ago

The duplicate insertion area was the key here, due to duplicates in the langtag.json source, this meant the dictionary was not completely loaded resulting in these other tests failing. In other words those failed tests where a side effect of the duplicate data error.

mcdurdin commented 1 week ago

Failing tests due to data issues in langtags.json, reported at:

Note that most of the failing tests are caused by a cascade of errors rather than these root causes.