AppleCoreText Issue with Kannada text

[MayuraVerma]

Defect Report

This is not an issue with Noto Sans Kannada Font, but this is an issue with iOS and MacOS, "AppleCoreText API". If you can please report it to Apple, It might traction with Apple Engineers.

Title

Kannada text is broken in Notes app in iOS and mac OS

Font

'NotoSansKannada-Regular.ttf'

Where the font came from, and when

part of iOS 13 and mac OS Catalina

OS name and version

iOS 13.2.2 (17B102) mac OS catalina 10.15.1 (19B88)

Application name and version

Notes app

Issue

These text is displayed correctly in "NSTextView" field, but breaks in "NSTextField" text field. Notes app and all forms with text editing uses "NSTextField". Open Notes app in iOS or mac OS, type the below text. ಕಜ್ನೋ ಕಜ್ನ್ಕೋ ಕಜ್ನ್ಕ್ರೋ ಕಜ಼್ನೋ ಕಜ಼್ನ್ಕೋ ಕಜ್ಫ಼ೋ ಕಜ್ನ್ಫ಼ೋ ಕಜ್ಜ಼್ಫ಼ೋ ಕಜ಼್ಜ಼್ಫ಼ೋ

Noto Sans Kannada font is breaking in iOS and mac OS with cluster of Kannada text

U+0CCD U+0CCD U+0CCA U+0CCD U+0CCD U+0CCB U+0CBC U+0CCD U+0CBC U+0CCD U+0CBC U+0CCA U+0CBC U+0CCD U+0CBC U+0CCD U+0CBC U+0CCB ### Screenshot > Actual text in Notes App while editing. "NSTextField"  > Expected text in email while reading. "NSTextView" 

[nizarsq]

Apparently this issue still reproducible, TextView looks ok, but breaks in TextField.

[MayuraVerma]

This is still not resolved in MacOS Big Sur 11.0 Beta (20A4299v)

[MayuraVerma]

Text: ಅಕ್ಟ್ಬೋಬರ್ ಅಕ್ಟ್ರೋಬರ್‌ ಅರ್ಕ್ಟ್ರೋಬರ್ ಅರ್ಕ್ಟೋಬರ್ ಅಕ್ಟ್ಬೋಬರ್ ಅಕ್ಟ್ಬೋಬರ್

ಅರ್ಕ್ಟೋಬರ್

This looks like font issue in Mac OS. Same text in Mac OS with Baloo Thamma, Lava Kannada, Sama Kannada and Tiro Kannada are rendering the correctly.

(Kannada MN, Kannada Sangam MN,) Noto Serif Kannada and Noto Sans Kannada have issues rendering the text in Mac OS.

It could be Mac OS doesn't support some open type feature which is used in Noto Sans/Serif. But other fonts are using different open type feature for the same combination of text, which is supported in Mac OS.

Either we should get Mac OS support this open type feature for Kannada, we should use supported open type feature.

Screengrab from Notes app in Mac OS Big Sur 11.0 Beta (20A4299v) ScreenGrab.zip

[omshivaprakash]

Continues to have issues in 11.2 Beta (20D5029f)

[MayuraVerma]

Sample text ಅಕ್ಟ್ಬೋಬರ್ ಅಕ್ಟ್ರೋಬರ್‌ ಅರ್ಕ್ಟ್ರೋಬರ್ ಅರ್ಕ್ಟೋಬರ್ ಅಕ್ಟ್ಬೋಬರ್ ಅಕ್ಟ್ಬೋಬರ್ ಅರ್ಕ್ಟೋಬರ್ ಕಜ್ನೋ ಕಜ್ನ್ಕೋ ಕಜ್ನ್ಕ್ರೋ ಕಜ಼್ನೋ ಕಜ಼್ನ್ಕೋ ಕಜ್ಫ಼ೋ ಕಜ್ನ್ಫ಼ೋ ಕಜ್ಜ಼್ಫ಼ೋ ಕಜ಼್ಜ಼್ಫ಼ೋ

Pattern Noto Sans Kannada font is breaking in iOS and mac OS with cluster of Kannada text Kannada consonant U+0CCD Kannada consonant U+0CCB Kannada consonant U+0CCD Kannada consonant U+0CCD Kannada consonant U+0CCB Kannada consonant U+0CBC U+0CCD Kannada consonant U+0CBC U+0CCD U+0CBC Kannada consonant U+0CCB Kannada consonant U+0CBC U+0CCD Kannada consonant U+0CBC U+0CCD Kannada consonant U+0CBC U+0CCB

This issue is observed only in NSTEXTFIELD not in NSTEXTVIEW

Few examples of these two fields

NSTEXTFIELD in macOS notes, edit field contacts, edit field Music app, edit tags Pages, comments field Finder, while creating/editing the file/folder name Safari, address bar, web form

NSTEXTFIELD in iOS notes, edit field contacts, edit field 3rd party apps, Twitter, tweet compose Youtube, comments, or any field while entering/editing text Whatsapp, composing messages

NSTEXTVIEW in macOS notes, preview of list of notes contacts, view contact list Music app, view music tags. Pages, main window, text box Finder, view folder/file name Safari, view web pages

NSTEXTVIEW in iOS notes, preview of list of notes contacts, view contact list 3rd party apps, Twitter, view tweet Youtube, view comments, or any other tags Whatsapp, view received messages

Radar reports FB6132087 FB6150858 FB5711873 FB6150832 FB6150787 FB7373231 FB7838782

[MayuraVerma]

Multiple issues in TextKit 2, same feedbacks are added to openradar

FB9484908 | TextKit2: invisible Kannada chandrabindu sign FB9484854 | TextKit2 Kannada text selection and cursor movement error 2 FB9484770 | Textkit2 linespacing issue FB9480360 | TextKit2 Cursor position with zero width joiner and Kannada letter FB9479036 | TextKit2 Kannada text selection and cursor movement error

[MayuraVerma]

Noto Sans Kannada is the system UI font in iOS and mac OS.

SOLUTION:

This issue can be resolved in font with precomposed for Kannada consonant and U+0CCA glyph in the font. This would reduced the glyph count by 2, similar to Tiro Kannada at https://github.com/TiroTypeworks/Indigo/tree/main/fonts/TiroKannada by @TiroTypeworks

Glyph count in Noto Sans Kannada: Kannada consonant and U+0CCB is broken down to Kannada consonant base + U+0CC6 + U+0CC2 + U+0CD5 : Glyph count 4 Kannada consonant and U+0CCA is broken down to Kannada consonant base + U+0CC6 + U+0CC2: Glyph count 3

Tiro Kannada: Kannada consonant and U+0CCB is broken down to (Precomposed: Kannada consonant base + U+0CC6 + U+0CC2) + U+0CD5 : Glyph count 2 Kannada consonant and U+0CCA is broken down to (Precomposed: Kannada consonant base + U+0CC6 + U+0CC2): Glyph count 1

Tiro Kannada font do not exhibit is this issue in iOS/macOS, because of its 2 less glyph to deal for TextKit.

Both Noto Sans Kannada and Noto Serif Kannada can overcome the glyph count limitation in iOS and macOS.

Background:

Glyph count limitation in TextKit (ver 1) can be found in below two videos.

WWDC 2021, watch from 7:00 https://developer.apple.com/videos/play/wwdc2021/10061 WWDC 2022, watch from 15:20 https://developer.apple.com/videos/play/wwdc2022/10090

TextKit2 was first introduced in 2020.

iOS 16/mac OS ventura which is slated to be released in Sep2022. TextKit2 is made default only in UITextView/NSTextView but developers can opt out, if they have compatibility issue. Still no word on when Texkit2 will be default and made in UITextField/NSTextField, UILabel/NSComboNox.

UITextField/NSTextField is the most used field in iOS/macOS for text entry, editing and viewing for paragraphs.

Speed at which, Apple is implementing TextKit2 and providing backwards compatibility to TextKit. It might take another 5-10 years, by the time, TextKit2 becomes default and mandatory text engine across all text fields in iOS and macOS.

I request Noto Fonts to please implement this solution at the earliest.

[MayuraVerma]

@marekjez86 @nizarsq Please review this and prioritize. Noto Sans Kannada is the system UI font in iOS and mac OS. Solution proposed above will resolve a long standing and annoying bug in Kannada font in iOS and macOS

[MayuraVerma]

Noto Sans Kannada vs Tiro Kannada in Mac OS Ventura

[simoncozens]

Thanks for this; there seem to be various messes here. I see in the Indigo bug there is the suggestion of attaching the candrabindu via mark positioning. I will work on this, but I have quite a bit of other things to go through first.

[MayuraVerma]

@simoncozens

you can find more info in the openradar. Kannada consonant with U+0CCB: This is a very bad bug. It affects every Kannada user, who uses Mac or iDevice. https://openradar.appspot.com/FB6132087

Solution is here. We need precomposed glyph for Kannada consonants and U+0CCA in Noto Sans Kannada Shaping engine adds U+0CD5 to U+0CCA to form U+0CCB. https://github.com/notofonts/kannada/issues/5#issuecomment-1161928210

ChandraBindu Sign: this is not a priority. https://openradar.appspot.com/FB9484908

[MayuraVerma]

iOS17 beta 1 resolves this issue. Please check and close.

[omshivaprakash]

Yes, it’s a big relief.

[simoncozens]

Finally. Thanks, Apple!