Add InputType 'insertReplacementText' for all spellcheck and substitution related actions

[chong-z]

We don't have InputType for spellcheck right now, assume we want one, should it be spellcheck or insertFromSpellcheck or something else?

[johanneswilm]

This has come up before. I believe last time we said that there may (some time in the future) be many different types of language checkers (grammar checker, spell checker, etc.), so we just created the general "replaceContent".

However, that isn't really our reality right now, and I wonder if those other testers wouldn't be JS based anyway, so I wouldn't be opposed to adding such an inputType.

As for the name of it: I guess it would always be an insertion, at times replacing existing content. Even if the correction is about deleting a character, it would presumably try to replace the entire word. So insertFromSpellcheck may make sense. On the other hand: Will this always replace existing content? In that case the name may be confusing. What will be used if someone tabs on a predicted word in a mobile OS keyboard, for example?

[chong-z]

This has come up before. I believe last time we said that there may (some time in the future) be many different types of language checkers (grammar checker, spell checker, etc.), so we just created the general "replaceContent".

If we add replaceContent won't it cause confusion since we already have insert* with getTargetRanges(), which should essentially do replacement as well?

As for the name of it: I guess it would always be an insertion, at times replacing existing content. Even if the correction is about deleting a character, it would presumably try to replace the entire word. So insertFromSpellcheck may make sense.

Yes. Chrome will replace the entire marked selection even if it's just inserting or deleting 1 character. I've also tested Safari and Firefox, which seems to do the replacement as well, and the styled text inside will lost. e.g. Right click and choose suggested word:

• 'paragraphk' => 'paragraph'

On the other hand: Will this always replace existing content? In that case the name may be confusing. What will be used if someone tabs on a predicted word in a mobile OS keyboard, for example?

If it's on mobile OS keyboard I would expect a composition to start, and the browser doesn't know if it's an IME spellcheck or normal composition. Or did I misunderstood something?

[johanneswilm]

This has come up before. I believe last time we said that there may (some time in the future) be many different types of language checkers (grammar checker, spell checker, etc.), so we just created the general "replaceContent".

If we add replaceContent won't it cause confusion since we already have insert* with getTargetRanges(), which should essentially do replacement as well?

Sorry, I should have mentioned how the replaceContent was later renamed to insertNonText.

On the other hand: Will this always replace existing content? In that case the name may be confusing. What will be used if someone tabs on a predicted word in a mobile OS keyboard, for example?

If it's on mobile OS keyboard I would expect a composition to start, and the browser doesn't know if it's an IME spellcheck or normal composition. Or did I misunderstood something?

I see. Yes, I guess due to it all being in an IME session, the JavaScript cannot really intervene a lot in this part anyway. Maybe it would make sense to add different types here some time in the future when IME does allow interventions.

So I guess insertFromSpellcheck would be the most consistent term, even if it seems slightly odd by itself.

[chong-z]

Resolution: Use inputType 'insertReplacementText' for all spellcheck and substitution related actions. https://www.w3.org/2016/09/22-webapps-minutes.html#resolution01

[alexelias]

This proposal seems technically infeasible on Android web browsers. The dominant form of spellcheck on Android is keyboard-app driven word replacement which appears to the UA as a simple composition update with no signal as to the reason. So I don't think we should be adding this.

[johanneswilm]

@alexelias I think there is a little misunderstanding here. During the composition, we only get beforeinput events with the inputType = "insertCompositionText". They cannot be canceled and we don't specify what type of IME interaction led to the change being made. So in your Android case, if a user composes a word and the IME helps with a T9 function to make that composition happen, it doesn't change the inputType of the beforeinput events during the composition.

The "insertReplacementText" is only meant for outside the IME. If Android never uses spellchecking outside IME, it doesn't ever have to trigger this. The list of inputTypes is a list of all the inputTypes that CAN be triggered, but no UA is obliged to cover all of them.

So I don't think we should change the decision we made, but maybe we need to add a note about predictive text as part of an IME composition so that others don't get confused by that.

[alexelias]

Why not just use inputType = "insertCompositionText" for the UA spellcheck changes instead of introducing a new platform-specific type? Using a single type for all types of spellcheck would help JS editors naturally interop across Android and desktop.

So in your Android case, if a user composes a word and the IME helps with a T9 function to make that composition happen

That's not really what I have in mind. On Android, if you have an already typed paragraph with a spelling mistake in one word: 1) When the user places their cursor over it, Android IME issues a composition start to underline the word. (In fact, Google Keyboard usually starts a word-spanning composition whenever any existing word comes under the cursor, spelling mistake or not.) It displays a list of spelling replacement suggestions in platform UI. 2) If the user taps on one of the spelling suggestions, the IME then issues a composition update to replace the word. 3) When the cursor moves away from the word, the composition ends.

This use case is quite analogous to the UA-implemented spellcheck features found on desktop browsers, so I think the events issued to JS should also appear mostly the same. And you haven't explained what is the additional value added from the point of view of a JS app from knowing that a change came from a desktop-specific-spellcheck as opposed to an IME.

[johanneswilm]

Using a single type for all types of spellcheck would help JS editors naturally interop across Android and desktop.

The goal is to have JS know more about what the user actually wants to do in terms of semantics instead of trying to guess what is going on after the fact. That's the main point of the beforeinput event.

Unfortunately, IME creates a number of problems because it's all non-standardized and cannot be canceled. So for IME, we currently try to isolate it as much as possible and give JS control over the final result. In the long-term, we hope to give the same kind of faingrained information and control around IME-based input as we can give for other input.

So the fact that Android currently only allows spellcheck this one way should not stop us from providing etter controls on desktop computers. For serious text writing, desktop computers continue to be highly important.

@alexelias btw, this issue has been resolved, and what you are discussing here is a subject that is somewhat different from what was discussed earlier. Could you please open a new issue for the issue you are discussing?

[johanneswilm]

That's not really what I have in mind. On Android, if you have an already typed paragraph with a spelling mistake in one word: 1) When the user places their cursor over it, Android IME issues a composition start to underline the word. (In fact, Google Keyboard usually starts a word-spanning composition whenever any existing word comes under the cursor, spelling mistake or not.) It displays a list of spelling replacement suggestions in platform UI. 2) If the user taps on one of the spelling suggestions, the IME then issues a composition update to replace the word.

Right, so it's happening inside an IME composition that we don't have any control over in JS.

And you haven't explained what is the additional value added from the point of view of a JS app from knowing that a change came from a desktop-specific-spellcheck as opposed to an IME.

Very briefly: insertCompositionText cannot be canceled, because it happens during IME. All the beforeinput events outside of IMEs can be canceled. So JS apps will treat insertCompositionText as something they just have to accept and cannot influence at the moment. But instead one could ask: Why not use 'insertText' for changes that come from the spell checker. This is the input type used for inserting text using the keyboard, so surely that would work for the spell checker as well. The answer to that question is: inserting text with the keyboard means that the caret position will change and the insertion happens necessarily near the caret. For spell checking, this doesn't have to happen. One can maintain the caret in one place and fix a misspelled word somewhere different.

I would also suggest you look at the part of the spec explaining what the point is of introducing the beforienput event is. You could also look at the IRC logs or the discussions on the email list over the past few years, or the open and closed github issues both here and in the "editing" repository.

[alexelias]

Very briefly: insertCompositionText cannot be canceled, because it happens during IME.

That's not a universal rule of all IME (possibly including Android IMEs, although we still need to investigate the implications of cancelling). We could be using the "cancelable" property of events dynamically to specify which events are cancelable or not, thus only remaining incompatible with a subset of IMEs, and aiming to increase cancelability coverage more and more over time. I added a comment on http://crbug.com/652397 for us to investigate the possibilities further.

But instead one could ask: Why not use 'insertText' for changes that come from the spell checker. [...] For spell checking, [...] one can maintain the caret in one place and fix a misspelled word somewhere different.

IME operations however can atomically choose cursor position in conjunction with the change. This argument is a good one for not merging with insertText only.

So the fact that Android currently only allows spellcheck this one way should not stop us from providing etter controls on desktop computers.

My issue with separating things this way is that web authors will naturally expect that Android spellchecks go through the event that the webspec says is for spellcheck, and they would be badly confused by a story that Android spellchecks are somehow not spellchecks and we have no power to ever fix that bug.

I'm sorry about coming in with late bomb-throwing feedback, I'll aim to express my concerns at earlier stages in the future.

[johanneswilm]

That's not a universal rule of all IME (possibly including Android IMEs, although we still need to investigate the implications of cancelling).

We are aware that some IMEs could possibly be canceled, but we decided to have it be non-cancelable in all cases in this first version.

IME operations however can atomically choose cursor position in conjunction with the change. This argument is a good one for not merging with insertText only.

Yes, for now we accept that we don't really have any control what happens or doesn't happen during IME. This is a compromise that was carefully negotiated over the past two years. Once the first version of this spec has shipped, we will have to discuss what will happen in the next version.

My issue with separating things this way is that web authors will naturally expect that Android spellchecks go through the event that the webspec says is for spellcheck, and they would be badly confused by a story that Android spellchecks are somehow not spellchecks and we have no power to ever fix that bug.

I don't think they will be confused. This is essential information that is very important for creating editors for desktop browsers and in the future hopefully we can get IME "under control", allow canceling, and then also change the types of beforeinput events emitted during IME. In the meantime we will have to live with this two-fold way of doing things:The JS has a lot of control during non-IME input, and limited control during IME-input.

Also, be aware that we are not targetting entry-level web developers who try to create a webpage with a comment box in an hour or two. Creating a JS editor is something that takes at least half a year, for a simple editor. Most working editors have been many years in the making. We are targeting maybe ten teams worldwide creating such editors. So when they spend that much time on creating the editor, they also have time to read through the spec a few times.

[alexelias]

OK, I withdraw my objection to this event type after all, for the following reasons: A) Android also has a (relatively obscure) system spellcheck feature. This behaves pretty similarly to the desktop spellchecks and could reasonably use this same type. B) Upon reflection, I don't think it's a such a great idea for the UA to interject itself temporarily as a fake IME in between the real IME and the JS for the purpose of spellcheck. This would particularly be problematic when a spellcheck replacement happens while an ongoing IME composition is still active.

So I'm fine after all with going forward with the plan as is, just with a comment that this doesn't apply to IME spellchecks. Thanks for your consideration.

[johanneswilm]

@alexelias Great!

The fact that you say that the IME on Android can be canceled would be very interesting to JS editor creators for the next version of the spec. Maybe we could start mapping what IMEs could theoretically be canceled/receive alternative input so that we can see whether we could give greater control over IME input to JavaScript the next version of the spec?

[alexelias]

Yes, I'm interested in empirically gathering results on feasibility of this. I'll let you know if I have interesting findings (whether in the direction that certain IMEs can tolerate cancellation/mutation with a given approach, or that they cannot).