x/text/unicode/cldr: Decoder fails to load cldr-common-41.0.zip

[dolmen]

What version of Go are you using (go version)?

$ go version
go version go1.18.2 darwin/amd64
$ go list -m golang.org/x/text
golang.org/x/text v0.3.7

Does this issue reproduce with the latest release?

Yes. golang.org/x/text v0.3.7

What did you do?

Using the latest CLDR release v41.0: http://unicode.org/Public/cldr/41/cldr-common-41.0.zip

    zip, err := os.Open(cldrArchivePath)
    if err != nil {
        log.Fatalf("%s: %s", cldrArchivePath, err)
    }

    cldrDecoder := &cldr.Decoder{}
    log.Println("Loading...")
    db, err := cldrDecoder.DecodeZip(zip)
    if err != nil {
        log.Fatalf("%s: %s", cldrArchivePath, err)
    }
    log.Println("success.")

Full code: https://github.com/blueboardio/cldr/blob/master/currency/currencies_gen.go#L73

What did you expect to see?

success.

(like with cldr-common-40.0.zip)

What did you see instead?

cldr-common-41.0.zip: supplemental-temp/coverageLevels2: missing identity element

[mknyszek]

CC @mpvl also.

[neild]

I attempted to fix this, and spent rather more time learning about CLDR than I intended. Documenting what I found so far:

• CLDR 41 has a supplemental-temp directory which needs to be handled in unicode/cldr.Decoder.Decode.
• The latest version of iana/assignments/language-subtag-registry has an entry where the three letter language code adp is deprecated in favor of dz. This results in the generator getting confused (internal/language/gen.go#814) while generating the lang table because it assumes a three-letter code is never deprecated in favor of a two-letter one. I think the fix is to generate this as adp\x00.
• CLDR 41 violates the specification of parent locales (TR 35 4.1.3) with two entries that change the base language code of a locale:

  <parentLocale parent="en_IN" locales="hi_Latn"/>
  <parentLocale parent="no" locales="nb nn"/>

  This confuses writeParents. I'm not sure what the correct fix for this is.

[dolmen]

Still happens with latest code (golang.org/x/text v0.6.0) and data (cldr-common-42.0.zip).

[dolmen]

I found a workaround for my use case: use a DirFilter:

cldrDecoder.SetDirFilter("main", "supplemental")