uniprop needs tests for all unicode properties

[samcv]

Emoji:

See: http://unicode.org/reports/tr51/#Data_Files for how they are determined (These are Boolean) Values stored here: http://unicode.org/Public/emoji/latest/emoji-data.txt If you hadn't seen this before, really great resource to check which symbols have a property:

http://unicode.org/cldr/utility/properties.html

• [x] Emoji
• [x] Emoji_Presentation
• [x] Emoji_Modifier
• [ ] Emoji_Modifier_Base
• [x] Emoji_All

  Emoji_Zwj_Sequences is a property of multiple codepoints together (we may need another routine to do this since it is a property of sequences of codepoints).

• [ ] Emoji_Zwj_Sequences

Numeric Properties

• [ ] cjkAccountingNumeric ; kAccountingNumeric
• [ ] cjkOtherNumeric ; kOtherNumeric
• [ ] cjkPrimaryNumeric ; kPrimaryNumeric
• [x] nv ; Numeric_Value

String Properties

• [x] cf ; Case_Folding
• [ ] cjkCompatibilityVariant ; kCompatibilityVariant
• [ ] dm ; Decomposition_Mapping
• [ ] FC_NFKC ; FC_NFKC_Closure
• [x] lc ; Lowercase_Mapping
• [ ] NFKC_CF ; NFKC_Casefold
• [ ] scf ; Simple_Case_Folding ; sfc
• [x] slc ; Simple_Lowercase_Mapping
• [x] stc ; Simple_Titlecase_Mapping
• [x] suc ; Simple_Uppercase_Mapping
• [x] tc ; Titlecase_Mapping
• [x] uc ; Uppercase_Mapping

Miscellaneous Properties

• [x] bmg ; Bidi_Mirroring_Glyph
• [x] bpb ; Bidi_Paired_Bracket
• [ ] cjkIICore ; kIICore
• [ ] cjkIRG_GSource ; kIRG_GSource
• [ ] cjkIRG_HSource ; kIRG_HSource
• [ ] cjkIRG_JSource ; kIRG_JSource
• [ ] cjkIRG_KPSource ; kIRG_KPSource
• [ ] cjkIRG_KSource ; kIRG_KSource
• [ ] cjkIRG_MSource ; kIRG_MSource
• [ ] cjkIRG_TSource ; kIRG_TSource
• [ ] cjkIRG_USource ; kIRG_USource
• [ ] cjkIRG_VSource ; kIRG_VSource
• [ ] cjkRSUnicode ; kRSUnicode ; Unicode_Radical_Stroke; URS
• [x] isc ; ISO_Comment
• [x] JSN ; Jamo_Short_Name
• [x] na ; Name
• [x] na1 ; Unicode_1_Name

  Name_Alias and Script_Extensions can hold multiple values. It is not yet determined how we will access them once they are added to some backend

• [ ] Name_Alias ; Name_Alias
• [ ] scx ; Script_Extensions

Catalog Properties

• [x] age ; Age
• [x] blk ; Block
• [x] sc ; Script

Enumerated Properties

• [x] bc ; Bidi_Class
• [x] bpt ; Bidi_Paired_Bracket_Type
• [x] ccc ; Canonical_Combining_Class
• [x] dt ; Decomposition_Type
• [x] ea ; East_Asian_Width
• [x] gc ; General_Category
• [x] GCB ; Grapheme_Cluster_Break
• [x] hst ; Hangul_Syllable_Type
• [x] InPC ; Indic_Positional_Category
• [x] InSC ; Indic_Syllabic_Category
• [x] jg ; Joining_Group
• [x] jt ; Joining_Type
• [x] lb ; Line_Break
• [x] NFC_QC ; NFC_Quick_Check
• [x] NFD_QC ; NFD_Quick_Check
• [x] NFKC_QC ; NFKC_Quick_Check
• [x] NFKD_QC ; NFKD_Quick_Check
• [x] nt ; Numeric_Type
• [x] SB ; Sentence_Break
• [x] WB ; Word_Break

Binary Properties

• [x] AHex ; ASCII_Hex_Digit
• [x] Alpha ; Alphabetic
• [x] Bidi_C ; Bidi_Control
• [x] Bidi_M ; Bidi_Mirrored
• [x] Cased ; Cased
• [ ] CE ; Composition_Exclusion
• [x] CI ; Case_Ignorable
• [x] Comp_Ex ; Full_Composition_Exclusion
• [x] CWCF ; Changes_When_Casefolded
• [x] CWCM ; Changes_When_Casemapped
• [x] CWKCF ; Changes_When_NFKC_Casefolded
• [x] CWL ; Changes_When_Lowercased
• [x] CWT ; Changes_When_Titlecased
• [x] CWU ; Changes_When_Uppercased
• [x] Dash ; Dash
• [x] Dep ; Deprecated
• [x] DI ; Default_Ignorable_Code_Point
• [x] Dia ; Diacritic
• [x] Ext ; Extender
• [x] Gr_Base ; Grapheme_Base
• [x] Gr_Ext ; Grapheme_Extend
• [x] Gr_Link ; Grapheme_Link
• [x] Hex ; Hex_Digit
• [x] Hyphen ; Hyphen
• [x] IDC ; ID_Continue
• [x] Ideo ; Ideographic
• [x] IDS ; ID_Start
• [x] IDSB ; IDS_Binary_Operator
• [x] IDST ; IDS_Trinary_Operator
• [x] Join_C ; Join_Control
• [ ] LOE ; Logical_Order_Exception
• [x] Lower ; Lowercase
• [x] Math ; Math
• [ ] NChar ; Noncharacter_Code_Point
• [ ] OAlpha ; Other_Alphabetic
• [ ] ODI ; Other_Default_Ignorable_Code_Point
• [ ] OGr_Ext ; Other_Grapheme_Extend
• [ ] OIDC ; Other_ID_Continue
• [ ] OIDS ; Other_ID_Start
• [ ] OLower ; Other_Lowercase
• [ ] OMath ; Other_Math
• [ ] OUpper ; Other_Uppercase
• [ ] Pat_Syn ; Pattern_Syntax
• [ ] Pat_WS ; Pattern_White_Space
• [ ] PCM ; Prepended_Concatenation_Mark
• [x] QMark ; Quotation_Mark
• [x] Radical ; Radical
• [x] SD ; Soft_Dotted
• [x] STerm ; Sentence_Terminal
• [x] Term ; Terminal_Punctuation
• [ ] UIdeo ; Unified_Ideograph
• [x] Upper ; Uppercase
• [x] VS ; Variation_Selector
• [x] WSpace ; White_Space ; space
• [ ] XIDC ; XID_Continue
• [ ] XIDS ; XID_Start
• [ ] XO_NFC ; Expands_On_NFC
• [ ] XO_NFD ; Expands_On_NFD
• [ ] XO_NFKC ; Expands_On_NFKC
• [ ] XO_NFKD ; Expands_On_NFKD

Total: 118 + 6 Emoji

Implementation specific properties

These are not official Unicode properties and should not have tests written for them. They are listed here for completeness.

• MVM_COLLATION_PRIMARY
• MVM_COLLATION_SECONDARY
• MVM_COLLATION_TERTIARY
• MVM_COLLATION_QC
• Numeric_Value_Numerator
• Numeric_Value_Denominator
• NFG_QC

[zoffixznet]

Can this be somehow generated instead of done manually?

[samcv]

To generate it I would probably have to write a script to read the UNIDATA files. Maybe it would make sense to generate them after we already have some tests in place. Because otherwise how do you test the generator?

More importantly, a large number of these are derived properties, that are set depending on how a bunch of other properties are set, so it would be quite hard to do.

Once we have tests for all of them it may be a good idea to get some generation in place though for certain properties that would be easy to generate for (that don't rely on a large number of other properties).

[flexibeast]

i'd like to help out with this; can i just start adding tests for properties which haven't yet been checked off?

[samcv]

Yep! See S15-unicode-information/uniprop.t for the properties

[flexibeast]

Great, thanks! Already looking at it. :-)

[flexibeast]

Okay, i've now made PR #222; only changed five properties, so you can make sure i'm on the right track.

[samcv]

Good thank you :)