Spell-check + Auto-Correct Support

MarcelKaeding commented 2 years ago

Comparing with Apple Notes where you can enable Spell-check and automatic correction of spelling mistakes. Ideally this would also be possible in SuperEditor text fields

Blquinn commented 1 year ago

Note that flutter's EditableText now supports spell check on iOS+Android https://github.com/flutter/flutter/issues/34688.

I'm not sure if that effects this ticket or not.

mattsrobot commented 1 year ago

How possible is this? Spell checking is such a good feature to look into, most people writing text on a daily basis need this.

matthew-carroll commented 1 year ago

Here's an update from a spike that @angelosilvestre put together (August, 2023)

Spell checking

As of today, Flutter only has support for spell checking on Android and iOS: https://github.com/flutter/flutter/issues/122433

Trying to enable spellcheck for macOS on Flutter's TextField results in the following exception:

Spell check was enabled with spellCheckConfiguration, but the current platform does not have a supported spell check service, and none was provided. Consider disabling spell check for this platform or passing a SpellCheckConfiguration with a specified spell check service.

So, implementing spell check would involve creating a plugin. We would need to create a platform channel to call the NSSpellChecker methods, like this, or UITextChecker, like this.

On the editor side, we would need to listen for document changes, call the spellcheck methods and highlight the text. Maybe we could use the reaction pipeline for that.

Autocorrection

Looking at the macOS text input plugin, there isn't any mention to autocorrection, so I suppose it's not supported. Setting autocorrect to true on a Flutter TextField doesn't seem to have any effect on macOS.

We could try a similar approach with platform channels and edit reactions to implement this, but IMO autocorrection should be handled automatically. I'm not aware if the NSTextInputClient used by Flutter supports this feature, but I would assume that if it does, autocorrection would generate replacement deltas, like on mobile platforms.

brian-superlist commented 2 months ago

Heya Matt, could you please revive the work on this one? More users are asking for this functionality and it's also going to be important with the new AI editing assistant changes apple is introducing in the next versions of macOS and iOS. Overall, this has higher prio for us than undo/redo.

KevinBrendel commented 2 months ago

If development on Undo/Redo is paused, please make it possible to deactivate it, so it doesn't serve as a memory leak and source of bugs in the meantime.

angelosilvestre commented 1 month ago

@matthew-carroll On Flutter, this is available only on Android and iOS. Flutter defines a SpellCheckService that is used by its textfield:

/// Determines how spell check results are received for text input.
abstract class SpellCheckService {
 /// Facilitates a spell check request.
 ///
 /// Returns a [Future] that resolves with a [List] of [SuggestionSpan]s for
 /// all misspelled words in the given [String] for the given [Locale].
 Future<List<SuggestionSpan>?> fetchSpellCheckSuggestions(
   Locale locale, String text
 );
}

Where the SuggestionSpan is defined as:

class SuggestionSpan {
  /// Creates a span representing a misspelled range of text and the replacements
  /// suggested by a spell checker.
  ///
  /// The [range] and replacement [suggestions] must all not
  /// be null.
  const SuggestionSpan(this.range, this.suggestions);

  /// The misspelled range of text.
  final TextRange range;

  /// The alternate suggestions for the misspelled range of text.
  final List<String> suggestions;

  /// Equals and hashcode ommited.
}

It isn’t related to the IME, so we can query suggestions for arbitrary pieces of text at any time. This doesn’t seem to require an open input text connection. The default implementation uses the flutter/spellcheck method channel, where the SpellCheck.initiateSpellCheck is invoked to fetch the suggestions.

Below are the spell-checking mechanisms used by Flutter on Android and iOS, and also the available mechanisms on other platforms.

Android

Android has a SpellCheckerSession, where an app can call getSentenceSuggestions to query the suggestions for a piece of text. A SpellCheckerSession is created by invoking TextServicesManager.newSpellCheckerSession . Android handles spell-checking for the whole input text in a single method call.

The results are delivered asynchronously in a callback.

References:

https://developer.android.com/reference/android/view/textservice/TextServicesManager

https://developer.android.com/reference/android/view/textservice/SpellCheckerSession#getSentenceSuggestions(android.view.textservice.TextInfo[], int)

iOS

iOS has an UITextChecker , which an app can call rangeOfMisspelledWordInString to find the text offset of a misspelled word, and then call guessesForWordRange with the returned range to get the suggestions.

The results are delivered synchronously.

Unlike Android, these methods handle a single word at a time, so in order to check an arbitrary paragraph of text, we need to call these methods multiple times, until we reach the end of the text.

References:

https://developer.apple.com/documentation/uikit/uitextchecker/1621029-rangeofmisspelledwordinstring?language=objc

https://developer.apple.com/documentation/uikit/uitextchecker/1621037-guessesforwordrange?language=objc

macOS

For mac, we can use NSSpellChecker , which is similar to UITextChecker. We call checkSpellingOfString to get the range of the first misspelled word, and guessesForWordRange to fetch the suggestions.

Like iOS, these methods must be called multiple times to fetch all the suggestions for a paragraph of text.

The results are delivered synchronously.

There are other properties/methods that might be useful:

NSSpellChecker.userReplacementsDictionary: returns the dictionary of word replacements, which can be used to implement automatic word substitution
NSSpellChecker.correctionForWordRange: returns a word for auto-correcting a misspelled word. There isn’t much detail about it on the docs.
NSSpellChecker.showCorrectionIndicatorOfType : it seems this method can be used to show a native user interface indicating that a correction is available. However, only one indicator may be displayed at a given time.

There is also mentions of substitutionsPanel and spellingPanel , which sounds like we can show native popovers, but the documentation doesn’t detail how to use those (maybe someone familiar with native macOS development might know how to do this).

References:

https://developer.apple.com/documentation/appkit/nsspellchecker?language=objc

https://developer.apple.com/documentation/appkit/nsspellchecker/1532957-checkspellingofstring?language=objc

https://developer.apple.com/documentation/appkit/nsspellchecker/1527419-guessesforwordrange?language=objc

https://developer.apple.com/documentation/appkit/nsspellchecker/1524925-userreplacementsdictionary?language=objc

https://developer.apple.com/documentation/appkit/nsspellchecker/1531542-correctionforwordrange?language=objc

Windows

On Windows, we can use the ISpellChecker to fetch spell-checking suggestions.

Windows checks the whole input with the ISpellChecker::Check method, returning multiple ISpellingError s.

Each ISpellingError has a CORRECTIVE_ACTION, which can be:

Get suggestions: The app should call ISpellChecker::Suggest to fetch the list of suggestions.
Replace: the app should replace the text without user interaction.
Delete: the app should prompt the user to indicate that the text should be deleted.

With the Windows spell-checking API we could implement auto-correction because of the replace action. I only found this concept on the Windows API. Flutter doesn’t have the concept of a corrective action in its SuggestionSpan .

There is a sample app which demonstrate how to use the API in https://github.com/microsoft/Windows-classic-samples/blob/main/Samples/SpellCheckerClient/README.md

The results are delivered synchronously.

References:

https://learn.microsoft.com/en-us/windows/win32/api/spellcheck/nf-spellcheck-ispellchecker-check

https://learn.microsoft.com/en-us/windows/win32/api/spellcheck/nn-spellcheck-ispellingerror

https://learn.microsoft.com/en-us/windows/win32/api/spellcheck/ne-spellcheck-corrective_action

https://learn.microsoft.com/en-us/windows/win32/api/spellcheck/nf-spellcheck-ispellchecker-suggest

Linux

It seems there isn’t any built-in spell-checking API for Linux, but there are open source libraries, like https://github.com/hunspell/hunspell.

Web

Unfortunately, it seems there isn’t any browser APIs to handle spell-checking, so third party packages/services would be needed.

Options

I see two options for us:

Contribute to Flutter implementing spellchecking for the missing platforms.
Creating a separate plugin for that.

Integration with SuperEditor

We should probably use reactions (maybe a SuperEditorPlugin?). When the user types, or pastes text, we perform spell-checking on the inserted words. With the results, we could create something like a TextSuggestionsAttribution to attach suggestions to a span of text. Then, we could do something similar to how we paint the composing region underline to paint spell-checking decoration below words. The suggestion popovers could query this attribution to get the suggestions to be displayed.

One thing to keep in mind is that, even on platforms which perform spell-checking synchronously, a spell-checking plugin will always be asynchronous, due to the use of method channels. So, we should think if/how this will affect our edit/react pipeline.

matthew-carroll commented 1 month ago

@brian-superlist - let's chat about the state of Flutter and spell check/autocorrection tomorrow on our call, given the research that @angelosilvestre did. I want to make sure we can get you the full scope of what Superlist users require.

matthew-carroll commented 1 month ago

@KevinBrendel I wouldn't say it's paused, but I'll try to add some controls to disable undo/redo for the time being. I'll also try to get a reasonable default policy added to avoid accumulating too much data.

KevinBrendel commented 1 month ago

@matthew-carroll Sounds good, thanks 👍

matthew-carroll commented 1 month ago

I've built a plugin that identifies spelling and grammar mistakes while typing. I'm not working on the UI/UX to apply suggested spelling corrections.

I took a look at a few other apps to see how they do it.

Notion doesn't suggest any corrections. You have to use their AI system to get help with spelling.

Obsidian doesn't show anything in the editor itself - but you can right click on a word to open a Mac context window, which lists some suggestions.

Apple Notes is shown in the following video - it's behavior is the most complicated I've seen.

https://github.com/user-attachments/assets/2dedda62-6bd4-43f4-873b-4be11aa20d28

Google Dos is shown in the following video.

https://github.com/user-attachments/assets/47f1ccdf-ba4f-4cfb-abbe-612a30ec5a3d

brian-superlist commented 1 month ago

Thanks, Matt. I chatted about this with Matej. Probably the best thing we can do is hook into super_context_menu to show suggestions within the context menu. Apple notes even does this if you select the whole word and right-click on the selected misspelled word.

Does the plugin you wrote also return the suggested spelling corrections as well?

superlistapp / super_editor