specify how compositionend works if the caret has been moved to a different element

[johanneswilm]

If one moves the caret/selection to a different element when compoisitionstart is triggered, browsers different on compositionend:

1. Chrome/Safari trigger the compositionend event on the element where composition takes place.
2. Firefox triggers the compositionend event on the element where the caret was at the time when the compositionstart event was triggered.

Additionally, browsers behave differently if the caret is moved to a different element when compositionend is triggered:

1. Chrome/Safari paste the composed characters in the place where the caret is moved to.
2. Firefox moves the caret to a different element and inserts no characters there by default. If the JS inserts the characters by means of execCommand/insertHTML, the endComposition event is triggered a second time.

The Chrome/Safari behavior is desired for the use case where one wants to have IME character insertion into the document stream be atomic -- either by doing composition somewhere else [1] or by using a Shadow DOM during composition [2].

[1] http://jsbin.com/wimuqowimu/15/edit?html,js,console,output (works in Chrome/FF) [2] http://jsbin.com/pacisugiwu/1/edit?html,js,output (only works in Chrome)

Edge behavior is close to FF behavior and Safari close to Chrome.

See also debate about this here: https://lists.w3.org/Archives/Public/www-dom/2015JulSep/0044.html

[kojiishi]

What use cases do you have in your mind? From the normal use cases, Chrome/Safari behavior is considered as a bug. Is it happening only when set from JS?

Consider re-conversion, that turns existing text into composition. If the composition is inserted into DOM, and if the selection was moved, the original text must be restored. In that case, moving composition text into new caret is undesirable.

[johanneswilm]

@kojiishi

What use cases do you have in your mind?

At a low level basically what you had proposed to do inside the browser with IME using the Shadow DOM to do IME input without changing the DOM. Only that @ojanvafai was strongly against that idea, so we found a solution whereby it should instead be possible to do things that way in JS code only [1].

At a higher level, the idea of having IME character commits come in "atomic commits" into the DOM. This would be helpful for among other things:

• Collaborative editing environments
• Creating a clientside undo history

From the normal use cases, Chrome/Safari behavior is considered as a bug

What do you mean by "From the normal use cases"?

The part about the Chrome/Safari behavior that I cannot see anything about in the spec is that it pastes the created characters into the editing host where the caret is placed when the compositionend event is triggered. The rest of it is according to spec, as far as I can tell.

Firefox, on the other hand, does not follow the spec in several ways, AFAICT. First of all, it should not trigger the event twice. Secondly, the spec reads "Event.target: focused element processing the composition". That is what Chrome/Safari do, while for Firefox the Event.target is the element that was focused BEFORE the composition started. Unless JS moves the caret somewhere else, there will be no difference between where the caret was before composition started and during composition started, which is likely why they didn't think of this case when they wrote the code.

Is it happening only when set from JS?

I am not sure what you mean. How would you set the caret anywhere else when compositionstart is triggered without using JS?

Consider re-conversion, that turns existing text into composition.

This is something you only do in Android and that is not used anywhere else, right?

If the composition is inserted into DOM, and if the selection was moved, the original text must be restored. In that case, moving composition text into new caret is undesirable.

I don't think I understand what you are trying to say here.

In general, I don't think that this re-conversion should be a problem for the JS solution [1] possibly with some minor modifications. Could you outline what the process is exactly for re-conversion, or do you have a link?

[1] http://jsbin.com/pacisugiwu/1/edit?html,js,output

[masayuki-nakano]

Secondly, the spec reads "Event.target: focused element processing the composition".

It should be a bug of UI Events.

That is what Chrome/Safari do, while for Firefox the Event.target is the element that was focused BEFORE the composition started.

In these days, modern IMEs queries content information before compositionstart. Therefore, moving focus at compositionstart is too late for native IMEs.

Firefox guarantees that a set of composition events must be fired on an element. It should be better for web developers because compositionend events may not be fired if focus is lost from listening event target unexpectedly.

[masayuki-nakano]

Firefox moves the caret to a different element and inserts no characters there by default. If the JS inserts the characters by means of execCommand/insertHTML, the endComposition event is triggered a second time.

I think that if the focused element was <input> or <textarea>, the composition string is committed on the element at moving focus?

IIRC, if it's a contenteditable element, it doesn't work fine due to our editor's bug (and it's very difficult to fix in current design...)

[johanneswilm]

Secondly, the spec reads "Event.target: focused element processing the composition".

It should be a bug of UI Events.

What do you mean? Do you mean that the UI Events spec should be changed? Or that Firefox has a bug?

That is what Chrome/Safari do, while for Firefox the Event.target is the element that was focused BEFORE the composition started.

In these days, modern IMEs queries content information before compositionstart. Therefore, moving focus at compositionstart is too late for native IMEs.

What are modern IMEs for you and what are native IMEs? I tried this on Chrome on Mac OS X with one of the Mac OS X built-in IMEs, and the Chrome example still works, which means that it's no problem to move the focus somewhere else during compositionstart for Chrome on Mac OS X. Also Safari is able to move the focus during compositionstart.

If there are UAs which start composition before compositionstart and therefore cannot move the selection somewhere else, I would say that those UAs have a bug and should be fixed.

Firefox moves the caret to a different element and inserts no characters there by default. If the JS inserts the characters by means of execCommand/insertHTML, the endComposition event is triggered a second time. I think that if the focused element was or , the composition string is committed on the element at moving focus?</p> </blockquote> <p>IIRC, if it's a contenteditable element, it doesn't work fine due to our editor's bug (and it's very difficult to fix in current design...)</p> </blockquote> <p>Could you guys look at the two pieces of example code I put together? :) </p> <p>And what is "our editor's bug"?</p> <p><input> and <textarea> do not matter much to us right now. In the Editing taskforce we are trying to fix editing in the browser, and having this work correctly is a key component.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/masayuki-nakano"><img src="https://avatars.githubusercontent.com/u/8064604?v=4" />masayuki-nakano</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>What do you mean? Do you mean that the UI Events spec should be changed?</p> </blockquote> <p>yes. It doesn't make sense compositionend (and/or compositionupdate) is fired on different element if the composition is committed after the element blurred. compositionstart should be fired on the focused element, but other composition events should be fired on the element which has the composition.</p> <blockquote> <blockquote> <blockquote> <p>That is what Chrome/Safari do, while for Firefox the Event.target is the element that was focused BEFORE the composition started.</p> </blockquote> <p>In these days, modern IMEs queries content information before compositionstart. Therefore, moving focus at compositionstart is too late for native IMEs.</p> </blockquote> <p>What are modern IMEs for you and what are native IMEs?</p> </blockquote> <p>Most Japanese IMEs are using surrounding text for prioritize the items in its candidate list at converting a word. For example, "saku" can be converted to "咲く" (bloom) or "割く" (tear). Then, the previous word is important for guessing what the user expected. If the previous word is "花が" (A flower)、"咲く" must be better. If the previous word is "布を" (cloth), "割く" must be better. So, this modern IME may query previous content of the insertion point. There is another scenario, user may commit composition with wrong candidate item. Then, user can use "Kakutei-Undo" (undo the previous commit). When user type Ctrl + Backspace, IME sets selection which covers the last committed string and restart the composition. Therefore, for Japanese IMEs, the preparation before composition start is very important.</p> <p>Similarly, as far as I know, Thai language's IME queries surrounding text and replace it with new composition string if it's necessary since Thai characters are combined one or more characters.</p> <p>Moving focus at compositionstart kills such features completely. So, such web application isn't useful.</p> <blockquote> <p>I tried this on Chrome on Mac OS X with one of the Mac OS X built-in IMEs, and the Chrome example still works, which means that it's no problem to move the focus somewhere else during compositionstart for Chrome on Mac OS X. Also Safari is able to move the focus during compositionstart.</p> </blockquote> <p>I think that some features which I mentioned above must not work in such cases.</p> <blockquote> <p>And what is "our editor's bug"?</p> </blockquote> <p>I meant that it is a bug of Firefox's editor.</p> <blockquote> <p><input> and <textarea> do not matter much to us right now. In the Editing taskforce we are trying to fix editing in the browser, and having this work correctly is a key component.</p> </blockquote> <p>contenteditable case is very complicated. If a document has two or more contenteditable elements, focus may be switched from one of them to another one. In such case, how to commit composition string at blur is difficult issue since all contenteditable elements share an editor instance in a document.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/ojanvafai"><img src="https://avatars.githubusercontent.com/u/1607171?v=4" />ojanvafai</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <p>@masayuki-nakano the use-case is to be able to move the focus into a shadow DOM inside the contentEditable. It would go in the same location in the DOM, so the IME would have the same text content to work with. It gives the web author a way of ensuring that the DOM changes from a composition only go into the DOM at compositionend as an atomic commit.</p> <p>Before this suggestion, the contentEditable spec required the UA to do this shadow DOM behavior. I objected to that and pointed out that authors can already do this in script.</p> <p>Even if you ignore shadow DOM, you could imagine putting it in a span inside the regular DOM, but in the same location. So the IME prediction would work the same, but the author would have full control over the DOM.</p> <p>Does that use-case sound legitimate to you? I believe the IME would function exactly the same in all these cases. It's true that if you don't know what you're doing, you can mess this up, but that's true today too. You can just as easily mess with the DOM in other ways that will break IME prediction.</p> <p>Regarding compositionend happening on the same element compositionstart happens on, why is that important? From a conceptual level, there are already plenty of instances where moving focus will cause the start/end events of a certain type to not fire on the same element. For example, changing focus during keydown causes keyup to fire on a different element. <a href="http://jsbin.com/kuvupalile/edit?html,console,output">http://jsbin.com/kuvupalile/edit?html,console,output</a></p> <p>There's a legitimate use-case for the behavior @johanneswilm is asking for.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/masayuki-nakano"><img src="https://avatars.githubusercontent.com/u/8064604?v=4" />masayuki-nakano</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>the use-case is to be able to move the focus into a shadow DOM inside the contentEditable. It would go in the same location in the DOM, so the IME would have the same text content to work with. It gives the web author a way of ensuring that the DOM changes from a composition only go into the DOM at compositionend as an atomic commit.</p> <p>Before this suggestion, the contentEditable spec required the UA to do this shadow DOM behavior. I objected to that and pointed out that authors can already do this in script.</p> <p>Even if you ignore shadow DOM, you could imagine putting it in a span inside the regular DOM, but in the same location. So the IME prediction would work the same, but the author would have full control over the DOM.</p> </blockquote> <p>Although, I don't understand what "the use-case" you said indicates.</p> <p>Thank you for your explain. I'm not familiar with "How Firefox treats shadow DOM", though, The last example is interesting. IIRC, starting Firefox 43, it won't commit/cancel composition even if selection is changed but the offset in the text isn't actually changed. So, the last example must work even on Firefox 43 for Windows or Linux. (IIRC, Firefox for Mac doesn't have this problem)</p> <p>FYI: the testcase <a href="http://jsbin.com/wimuqowimu/15/edit?html,js,console,output">http://jsbin.com/wimuqowimu/15/edit?html,js,console,output</a> won't work fine on Firefox 41 for Windows, unfortunately. Firefox 41 starts to support TSF on Windows. TSF is very rich framework, but very sensitive. For example, TSF doesn't allow to happen selection change/text change in focused editor at composition start. However, Firefox caches the previous contents of focused editor and lies TSF when it tries to query the content. If TSF-aware application returns unexpected content, TSF or some TIP (IME for TSF) may stop working :-( Anyway, I'll try to fix this bug latter...</p> <blockquote> <p>Does that use-case sound legitimate to you? I believe the IME would function exactly the same in all these cases. It's true that if you don't know what you're doing, you can mess this up, but that's true today too. You can just as easily mess with the DOM in other ways that will break IME prediction.</p> </blockquote> <p>If you said about <a href="http://jsbin.com/wimuqowimu/15/edit?html,js,console,output">http://jsbin.com/wimuqowimu/15/edit?html,js,console,output</a> , this wants more hack for Firefox developers... I'm not sure if this can be fixed easy.</p> <blockquote> <p>Regarding compositionend happening on the same element compositionstart happens on, why is that important? From a conceptual level, there are already plenty of instances where moving focus will cause the start/end events of a certain type to not fire on the same element.</p> </blockquote> <p>The main reason of that is, some web apps may work stateful with compositionstart/compositionend. For example, if web apps need to handle input only when the focused editor doesn't have composition, they need to make a flag to store if there is composition with compositionstart and compositionend events. If such web apps fails to listen one of them, their input event handler won't work as expected. For making such web apps stateless, I suggested isComposing attribute to InputEvent, but the browsers except Firefox have not supported it yet.</p> <p>Additionally, I believe that compositionstart, compositionupdate and compositionend are events of existing composition. Therefore, I believe that it makes sense the events fired on an element which includes the composition.</p> <p>If a browser doesn't commit composition even when the element loses focus but the composition is committed by other reason (e.g., destroying the element), why should composition events be fired on focused element? If they are always fired on an element which has composition, such scenario can be fixed simply.</p> <blockquote> <p>For example, changing focus during keydown causes keyup to fire on a different element.</p> </blockquote> <p>Yeah, right. But I feel that it makes sense that keyboard events are always fired on focused element because a focused element is a receiver element of keyboard event at the time. The difference of composition events are, the events are fired for notifying web apps of the composition state changes. Therefore, an element which has composition string should receive it.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/johanneswilm"><img src="https://avatars.githubusercontent.com/u/595512?v=4" />johanneswilm</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>FYI: the testcase <a href="http://jsbin.com/wimuqowimu/15/edit?html,js,console,output">http://jsbin.com/wimuqowimu/15/edit?html,js,console,output</a> won't work fine on Firefox 41 for Windows, unfortunately. Firefox 41 starts to support TSF on Windows. TSF is very rich framework, but very sensitive. For example, TSF doesn't allow to happen selection change/text change in focused editor at composition start. However, Firefox caches the previous contents of focused editor and lies TSF when it tries to query the content. If TSF-aware application returns unexpected content, TSF or some TIP (IME for TSF) may stop working :-( </p> </blockquote> <p>So it is currently working, but you are breaking it in the next version of Firefox? That does not sound like a step in the right direction. The spec says nothing about modifying the selection being forbidden as part of compositionstart, so that sounds as Firefox is planning to break with the spec?</p> <blockquote> <p>Anyway, I'll try to fix this bug latter...</p> </blockquote> <p>Ok, so you agree that this is a bug and you will fix it?</p> <blockquote> <p>If you said about <a href="http://jsbin.com/wimuqowimu/15/edit?html,js,console,output">http://jsbin.com/wimuqowimu/15/edit?html,js,console,output</a> , this wants more hack for Firefox developers... I'm not sure if this can be fixed easy.</p> </blockquote> <p>Not sure what you are saying here. This is the same example that you will be breaking in Firefox 41 on Windows, unless you fix it, that you mention above.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/masayuki-nakano"><img src="https://avatars.githubusercontent.com/u/8064604?v=4" />masayuki-nakano</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>The spec says nothing about modifying the selection being forbidden as part of compositionstart, so that sounds as Firefox is planning to break with the spec?</p> </blockquote> <p>Not planning, but if spec would be changed to so, Firefox implementation become much simpler... Sigh.</p> <blockquote> <p>Ok, so you agree that this is a bug and you will fix it?</p> </blockquote> <p>Yeah, if I can. Full support of TSF is very big project for web browsers because requirements (limitations) of TSF may not conform to some web standards. Therefore, Firefox including a lot of "fake" between TSF and DOM tree.</p> <p>FYI: Chrome supports TSF with transitory context, that means to support partially, additionally it's not used in desktop mode in default settings. IE11 also supports TSF, but I'm not sure if it's using transitory context.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/kojiishi"><img src="https://avatars.githubusercontent.com/u/1274377?v=4" />kojiishi</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <p>I still don't see the connection of what you're trying to do and the high level goals.</p> <p>Fixing collaborative editing environments on browsers was the motivation of the Shadow DOM discussion, so I agree with the goal. But client-side Shadow DOM library will not help the goal as far as I understand. I don't know about the other goal; a clientside undo history, so I can't comment on it.</p> <p>@ojanvafai knows Blink far much better than I do, so if he's against, I'm happy to follow. There should be side effects by making IME transparent or opt-in, I have to admit that I don't know all of them yet, and he should know them in depth.</p> <p>So, if all other vendors are happy, and JS developers are happy, it just means that collaborative editing on browsers must support IME explicitly, or they'll be broken, as current all offerings are. It's not a disaster.</p> <p>Back to the original topic, I'm still not convinced that defining the behavior of moving selections during compositions is a good thing. IME is still evolving to make better UX, and platforms do it differently. Android relies on it heavily, and OS X is going to change the UX in the next version. I'd like to avoid defining it so that we can allow platforms and IME to evolve further.</p> <p>It may be possible to define it without limiting platforms and IME evolutions, but I suspect it's not an easy task. If it's really desired, I'd like to understand the use case better, and how defining it can help the use case.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/johanneswilm"><img src="https://avatars.githubusercontent.com/u/595512?v=4" />johanneswilm</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>I still don't see the connection of what you're trying to do and the high level goals.</p> </blockquote> <p>What part is unclear to you? Have you looked at the two examples linked above? They contain instructions on how they are to function and you can see the use case from them.</p> <blockquote> <p>Fixing collaborative editing environments on browsers was the motivation of the Shadow DOM discussion, so I agree with the goal. But client-side Shadow DOM library will not help the goal as far as I understand.</p> </blockquote> <p>Can you elaborate a bit about that? Currently this is working quite well in Chrome Desktop at least on Mac OS X and Linux (on Android the compositionend event seems to be triggered immediately after compositionstart takes places). So I don't understand what you are trying to say with " client-side Shadow DOM library will not help the goal ". It's using the browser's native Shadow DOM implementation (no polyfill or clientside library). Shadow DOM isn't yet available everywhere, so when it isn't available, one can use the second example code.</p> <blockquote> <p>Back to the original topic, I'm still not convinced that defining the behavior of moving selections during compositions is a good thing. IME is still evolving to make better UX, and platforms do it differently. Android relies on it heavily, and OS X is going to change the UX in the next version. I'd like to avoid defining it so that we can allow platforms and IME to evolve further.</p> </blockquote> <p>It is a pity that you weren't at the editing taskforce F2F meeting. The current spec allows for what we need. Now you are talking about changing the spec to disallow it? That does not sound like a good way forward, and if the goal was to get first-class IME support, this seems to instead just make it rather impossible to give IME in many webbased editors.</p> <p>Do you have any alternative suggestion as to how to obtain the same result? </p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/johanneswilm"><img src="https://avatars.githubusercontent.com/u/595512?v=4" />johanneswilm</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <p>btw, if you want to change the compositionstart event to be triggered at a time when composition has started already to the point where it is not possible to change the selection, maybe you could introduce a beforeCompositionstart event that is triggered before anything else takes place so that the caret can still be moved and which can be canceled to prevent the composition from taking place at all?</p> <p>In order not to break anything, it would probably be better to just introduce it later.</p> <p>Another thought may be to fix TSF itself. If the TSF IME works considerably different than other IMEs, and web browsers are one of the most common Programs people use all day, maybe Windows 10's TWSF should be fixed instead?</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/kojiishi"><img src="https://avatars.githubusercontent.com/u/1274377?v=4" />kojiishi</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <p>Can you explain what issues there are in the current collaborative editing on browsers, and how your example can help solving those issues?</p> <p>The idea Ben and I were discussing was an early stage of the discussion to make IME inputs completely transparent, similar to what IME level 2 API on Windows does. If we discard the idea, there are several things apps must do to handle IME. Simple apps may need no-to-few, but collaborative editing might need several. The Shadow DOM-like building block was one part of making IME transparent. If it's not transparent, it looks to me that whether composition occurs within Shadow DOM or not isn't a big issue.</p> <p>If you have clear understanding of what the issues are, and seeing connections between your code and those issues, that's ok. I'm sorry if I'm wrong, but I wonder we (including me) don't have clear understanding on them, no?</p> <p>If you do, and moving compositions along with selection is a required feature to solve the issues, I'm ok to have that. Me not having an understanding isn't a blocking factor.</p> <p>But please keep in mind that the behavior you described has been considered as a very bad IME UX for a long period of time. Also it's likely to break re-conversion as mentioned above. We probably have to make sure it's allowed only when JS really needs it, but the behavior never be exposed otherwise.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/johanneswilm"><img src="https://avatars.githubusercontent.com/u/595512?v=4" />johanneswilm</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <p>@kojiishi: could you and @ojanvafai have a talk about what is and should be possible with Chrome? So far it seems he dismissed your idea of the transparent handling and you dismiss his idea of doing it through a JS controlled ShadowDOM. You guys know best what your code can be made to do without eating lots of resources. The F2F decided to go with this model which ojanvafai suggested, on the condition that IME through ShadowDOM would be possible in JS space. Ojanvafai briefly tested it during the meeting.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/kojiishi"><img src="https://avatars.githubusercontent.com/u/1274377?v=4" />kojiishi</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <p>@johanneswilm: hmm...looks like we're not understanding very well each other, wonder how I can explain it better.</p> <p>@ojanvafai dismissed the idea of transparent handling is fine with me, as commented above. I was actually expecting such discussions to see if making transparent is really feasible. If we don't go that route, what I'm trying to say is I think it's better to go other routes than transparent handling. This is nothing against what @ojanvafai said.</p> <p>Ben left the TF, and I no longer need compositions in Shadow DOM. If someone else needs it, that's fine. My question is who needs it, and for what purposes? If it's for collaborative editing, how it solves?</p> <p>The purpose of the question is, moving compositions along with selection could be costly, in current code and/or in possible future problems, so I'd like to make sure that compositions in Shadow DOM has a clear goal and benefits.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/johanneswilm"><img src="https://avatars.githubusercontent.com/u/595512?v=4" />johanneswilm</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <p>OK, once I find a computer, I will try to sketch it. Basically, content editable editors have gone through several generations, and it seems like the paradigm for the upcoming generation is to have three layers of the document:</p> <ol> <li>What can be seen in the browser.</li> <li>What is in the DOM.</li> <li>The document represented in a JavaScript object that can be handed around as Json.</li> </ol> <p>One layer of the JS code keeps 2 and 3 in sync. To do this job as easy as possible, it is important that any change to either one can be represented as an atomic commit that can be translated between 2 and 3 without ambiguity. The list of these commits can be stored to replay the document history, or by exchanged and run through an Operational Transformations (OT) handling mechanism for collaborative editing. 3 can be stored at any time, also during a composition, so that this stored version later can be recreated as 2 in a meaningful way (not with half a composition done or alike). IME input does not easily fit into this model, because the commits are not atomic and the difference between normal input mode and composition input is created. That's why it would be an advantage to be able to keep IME input handling at level 1 and only make the constructed characters hit level 2 and 3 once this can be done in forms of an atomic commit (ie when composition is done). To do this in a way that is transparent to the user, usage of the Shadow DOM as shown in my example would be helpful.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/johanneswilm"><img src="https://avatars.githubusercontent.com/u/595512?v=4" />johanneswilm</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <p>Also, if JS can stop any direct change to level 2 by means of a beforeInput/beforeEdit event, that means the processing of user input can be simplified: instead of having to diff 2 with an earliervversion to find out what has changed, one can catch the intended change in the before event, do the required changes at level 3, then translate changes at level 3 to changes at level 2 and update 2. In terms of inline IME editing in Shadow Dom, the user would do his IME at input level 1. Once characters have been constructed, the requested changes land at level 3, etc. .</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/johanneswilm"><img src="https://avatars.githubusercontent.com/u/595512?v=4" />johanneswilm</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <p>@kojiishi: How about this? This first model is a basic sketch of how an editor handling IME could be structured as of today in browsers that can handle it (Chrome). The advantage is that one can keep changes below what is shown on the screen atomic and with that create a reasonably well-structured editor. But notice also that within the DOM representation of the document (red) there are arrows going both ways: so sometimes the JS needs to update the DOM because the document has changed or the IME in the Shadow DOM has finished creating some characters. This means one needs to introduce a more complex system of changing the DOM and monitoring user-originated changes to it, which is likely to end up being rather complex and possibly never quite perfect. </p> <p><img src="https://cloud.githubusercontent.com/assets/595512/9677485/dd889cb6-52da-11e5-9f86-632695f22cdd.png" alt="model1" /></p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/johanneswilm"><img src="https://avatars.githubusercontent.com/u/595512?v=4" />johanneswilm</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <p>More realistic for such an editor is therefore in many cases this model. It cuts out some, but not all of the complexity, but does so by cutting out IME altogether. A variant of this model is probably more or less what you can find in many of the best editors that exist today.</p> <p><img src="https://cloud.githubusercontent.com/assets/595512/9677489/e7e692c6-52da-11e5-93c2-e8bec2a13513.png" alt="model2" /></p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/johanneswilm"><img src="https://avatars.githubusercontent.com/u/595512?v=4" />johanneswilm</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <p>With access to the new contenteditable modes and the beforeInput/Edit event, one could instead have something such as model 3. Here only the changes in the JavaScript object version of the document need to be translated to DOM changes, not the other way round. This should -- at least in theory -- make it somewhat easier to create an editor that does not suffer from having to sync in both directions. btw, the cEmode of the editable element in the DOM does not have to be cE=caret, but could also be a cE=typing, with all possible actions canceled by means of preventDefault. If we start out by only having cE=typing, this is probably what users will do.</p> <p>Of course this is an incredible simplification of an editor that will be tens- to hundreds of thousands of lines of code, but for most of the basic editing operations, I think this should more or less capture it.</p> <p>The reason why the IME input is treated separately from other type of input is that at the JS level we still will have very little control over the IME: once we enter a composition, we cannot stop it from making direct changes to the part of the DOM or Shadow DOM where the caret is placed, and we need to allow it to delete characters on its own, also with no way to cancel it. Keeping that kind of input isolated in the Shadow DOM until we have a finished, atomic commit that the JS code can decide what to do with, seems to be the most obvious way to isolate it from everything else in a meaningful way.</p> <p><img src="https://cloud.githubusercontent.com/assets/595512/9677492/f41c4504-52da-11e5-8957-cd6a2ce969b3.png" alt="model3" /></p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/kojiishi"><img src="https://avatars.githubusercontent.com/u/1274377?v=4" />kojiishi</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <p>Thank you for having this deep explanation, I didn't mean to ask you to do this much work, but having these models clearly help our discussions so much.</p> <p>IIUC, model 3 is a future thing, so let's focus on model 1/2.</p> <p>If your document model in JS does not want to incorporate composition strings, I understand the JS editor can import your polyfill to suppress DOM change events until composition is committed.</p> <p>On the other hand, if the JS editor wants the same effect without the polyfill, having a filter in the event handler should do.</p> <p>By using the polyfill, the filter may go down to 1 line, but the JS editor must still handle:</p> <ol> <li>Browsers without Shadow DOM support. This may go zero in years though.</li> <li>You can't create Shadow DOM against text nodes, so if the caret is in <code><span>|text<b>bold</b></span></code> either the polyfill or the JS editor must change DOM before composition starts.</li> <li>And as we discuss here, you need browsers to have a mode for this to work.</li> </ol> <p>Isn't having a filter in the event handler much easier?</p> <p>There are other things the JS editor must do to support IME for collaborative editing support. Suppressing composition event is one part. Another likely part is after OT (in your term) was received by other users, how to properly replay it on the other users' browser while s/he is typing compositions. If the editor moves selections to replay (I suppose all editors do this today,) it'll break typing experiences either by committing the composition or moving composition. This is probably larger issue than suppressing events. I guess save and restore may solve, but I'd like to hear from the real developers doing the work.</p> <p>And probably there are more that neither you nor I are aware of today.</p> <p>So my point is that, if an JS collaborative editor has to spend, say, 1 month of dev work to support IME in several functions of the editor anyway, having a filter in the event handler looks more reasonable to me than the Shadow DOM, especially when the polyfill is not perfect as mentioned above.</p> <p>But for the higher level of discussions, I'm not a JS developer, so which looks more reasonable to me isn't really the decision factor. If you and many others are really using it, I'm ok with your proposal. Only that the behavior you described could be harmful for other cases that we'll need a mode.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/johanneswilm"><img src="https://avatars.githubusercontent.com/u/595512?v=4" />johanneswilm</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <p>@kojiishi Sorry, I didn't see your answer until now.</p> <blockquote> <p>IIUC, model 3 is a future thing, so let's focus on model 1/2.</p> </blockquote> <p>Well, models 1 and 2 is kind of what JS editor developers are trying currently. The lack of support for IMEs in editors that are not primarily made for IME is probably exactly because of the problems they give. The idea here was to show why 1/2 are problematic, and why we have spent the past few years lobbying the browser makers and the last year creating a specification which will allow for model 3 and similar.</p> <blockquote> <ol> <li>Browsers without Shadow DOM support. This may go zero in years though.</li> </ol> </blockquote> <p>Well, for 1, there is the possibility to add a an area near where they are typing where the characters go during composition. That's the second polyfill. I know, you will say that is bad UI, but it's working and it's up to the browser makers to implement ShadowDOM so their users get a better user experience.</p> <blockquote> <ol> <li>You can't create Shadow DOM against text nodes, so if the caret is in <code><span>|text<b>bold</b></span></code> either the polyfill or the JS editor must change DOM before composition starts.</li> </ol> </blockquote> <p>Check the source code of the polyfill. It is already handled. It simply takes parent element node, adds a copy of all sibling nodes of the text node including hte text node itself and adds them to the shadow root, then discards all of them when done.</p> <blockquote> <ol> <li>And as we discuss here, you need browsers to have a mode for this to work.</li> </ol> </blockquote> <p>We need browsers as a minimum to stick to the current specs and nor worsen them, yes. But this is true for everything. We cannot show videos in the browsers if all the browser vendors suddenly decide to remove video playing features, etc. .</p> <blockquote> <p>If the editor moves selections to replay (I suppose all editors do this today,) it'll break typing experiences either by committing the composition or moving composition.</p> </blockquote> <p>No, why would you move the selection for that? Moving the selection in order to update the DOM makes little sense in general, but there may be browser bugs that people are working around that I am unaware of. What editor are you thinking of? Moving the selection to do changes to the DOM is not a good idea. There may be some other things you should also need to fix in browsers, but that's not a good excuse for breaking the thing that we just found out are currently possible with IME composition to get a little bit closer to editing sanity. This becomes even more interesting once one can actually really prevent any other DOM changes to take place in the editor so one can rely on only the JS handling it.</p> <blockquote> <p>Another likely part is after OT (in your term) was received by other users, how to properly replay it on the other users' browser while s/he is typing compositions.</p> </blockquote> <p>Changes to the DOM could continue to happen in the background. The user will not see those changes that are made within the same lowest level element node as where the user is typing, but everything else is there. Of course, there is a chance that the changes may cause the Shadow DOM to be destroyed, for example by removing the element that holds the root.In that case the composition will be ended abruptly. But that is the same as when people play "editing wars" using Latin language keyboard and delete the text section where the other user has their caret. It's not that big a problem in real life.</p> <blockquote> <p>So my point is that, if an JS collaborative editor has to spend, say, 1 month of dev work to support IME in several functions of the editor anyway, having a filter in the event handler looks more reasonable to me than the Shadow DOM, especially when the polyfill is not perfect as mentioned above.</p> </blockquote> <p>Ok, but then the polyfill does handle the issue you mention. And not having the polyfill creates the many problems as seen above. Trying to handle partially composed characters and distribute them between users may initially be 2-6 months of work for a good programmer, but it is also a complicating factor that will impact many other future developments within the editor. Not only does one need to keep track of who is typing what, one also needs to make sure that only the user themselves can delete their own partially composed characters. It's just a nightmare to manage, and that's why it's being done so little. </p> <p>So really, I think it's really crucial you A) do not break this or B) give us another solution whereby we can get IME input atomically. The alternative is that you spent a lot of time making the browser work with the newest and finest IME, but almost none of the webbased editors will be able to handle any of them.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/masayuki-nakano"><img src="https://avatars.githubusercontent.com/u/8064604?v=4" />masayuki-nakano</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <p>I still believe that a set of composition event (compositionstart -> compositionupdate * n -> compositionend) should be guaranteed that they are fired on a same element which is focused at compositionstart.</p> <p>First, compositionstart event should be computed the its target as focused (editable) element.</p> <p>Next, <em>if</em> compositionstart event handler moves focus and the browser starts composition on the new focused element, compositionend should be fired on the old focused element first, then, compositionstart and following composition events should be fired on the new focused element.</p> <p>Another possible cause is, if compositionupdate or input event handler moves focus and the browser continues the composing process in the new focused element, the old composition string should become empty string (or restore selected string which was removed by composition?), then, compositionupdate (if necessary) and compositionend should be fired on the old focused element and compositionstart and compositionupdate (if necessary) should be fired on the new focused element.</p> <p>This composition event process can support the stateful web apps with a pair of composiitonstart and compositionend. And even in new focused editor, a set of composition events are guaranteed.</p> <p>In conclusion, I believe that UIEvents need to change the composition event target as:</p> <p>compositionstart: focused element compositionupdate: an element which compositionstart event fired on. compositionend: an element which compositionstart event fired on.</p> <p>And the above behavior should be explained in UIEvents (Although, we've not had agreements if composition should be moved to new focused element, but for existing browsers, this behavior should be defined).</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/chandarsunderraj"><img src="https://avatars.githubusercontent.com/u/7701080?v=4" />chandarsunderraj</a> commented <strong> 7 years ago</strong> </div> <div class="markdown-body"> <p>facing this issue with Safari 10.0.1 & Chrome 56. when changing the keyboard to "Hiragana" Japanese. In first textarea if the focus interrupted before the compositionend ie without pressing enter to kill the composition. and pressing enter in second textarea the content typed in first textarea appears.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/marijnh"><img src="https://avatars.githubusercontent.com/u/144427?v=4" />marijnh</a> commented <strong> 5 years ago</strong> </div> <div class="markdown-body"> <p>Not sure how dead this is, but since it's still open I guess my input might at some point be useful.</p> <p>I (editor implementer, ProseMirror and CodeMirror) do <em>not</em> in fact want to isolate IME-in-progress from my document model. As in, I want IME changes to be treated like normal changes and have my user interface respond to them right away, not just when the composition finishes. The isolating model is what CodeMirror started with, and users found it surprising that things like autocompletion or context-dependent menus lagged behind during composition (reflecting the pre-composition state all the way until the composition was committed).</p> <p>So my problem is not one of needing a hack to move the selection somewhere at compositionstart, but rather of being able to touch the DOM around a composition without canceling/clearing the composition. I'd like to be able to continue highlighting syntax or doing other kinds of styling during composition, which may involve adding an inline parent node around the text at the cursor or moving it into a new parent, or splitting it in two, and then resetting the selection to the corresponding position in the changed DOM.</p> <p>I realize it's not likely that this'll be possible on any kind of short term, but ideally, for me, browsers would be 'robust' around DOM changes during composition—even if the DOM is modified and the selection changed, if the <em>textual</em> content around the new selection corresponds to what it used to be before the script messed with the document, a composition should be able to continue. That does raise some questions (like false positives when the selection was actually moved but its new context resembles the old) and might require new APIs, so consider it just a crude sketch of what I'd need.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/Dalzhim"><img src="https://avatars.githubusercontent.com/u/11513299?v=4" />Dalzhim</a> commented <strong> 4 years ago</strong> </div> <div class="markdown-body"> <p>Not sure how dead this is either, but since it's still open I guess my input might also be useful at some point. ;)</p> <p>There are two definitions in the current draft which I deem underspecified:</p> <ol> <li>composition session</li> <li>active text passage</li> </ol> <p>The reason for which I think they are underspecified is because Android has an implementation of composition sessions which is wildly different from all other platforms, and the current specification is insufficient to understand correctly what is happening during that composition session. Is the Android implementation considered compliant? With the current wording, I'd say it is. But the current situation is certainly not ideal.</p> <h1>1. composition session</h1> <p>A composition session is currently defined as starting with a <code>compositionstart</code> event, followed by one or more <code>compositionupdate</code> events and ending with a <code>compositionend</code> event. This seems underspecified in two different ways which lead to Android being different than every other platform I am aware of.</p> <h2>1.1. When to issue <code>compositionstart</code></h2> <p>Should a composition session start when the first change occurs, or is it sufficient to display an IME for existing text? As an example, on Android, placing the caret inside a word starts a composition for the word touched by the caret.</p> <h2>1.2. When to issue <code>compositionend</code></h2> <p>Can a composition session span multiple disjoint ranges? Currently, on Android, placing the caret inside a word, inserting one character and finally moving the caret inside a different word is all part of one single composition session. On other platforms, a composition session would start only when the character is entered, and moving the caret to a disjoint location would end the composition without starting a new one.</p> <h1>2. active text passage</h1> <p>The definition of the <code>componentupdate</code> event is the only place where the notion of an <code>active text passage</code> is mentioned. It does suggest that Android was taken into consideration as placing the caret inside a word will make the whole word part of that <code>active text passage</code>. Unfortunately, wordarounds must be used by client javascript code in order to reconstruct that range as it is never provided by the event.</p> <h1>Potential solution</h1> <p>Without requiring major changes to the current implementations, I believe client javascript code could handle composition events better if both <code>compositionstart</code> and <code>compositionupdate</code> events provided the range associated with the <code>active text passage</code>. This way, both behaviors unique to Android could be reasonably managed without brittle workarounds.</p> </div> </div> <div class="page-bar-simple"> </div> <div class="footer"> <ul class="body"> <li>© <script> document.write(new Date().getFullYear()) </script> Githubissues.</li> <li>Githubissues is a development platform for aggregating issues.</li> </ul> </div> <script src="https://cdn.jsdelivr.net/npm/jquery@3.5.1/dist/jquery.min.js"></script> <script src="/githubissues/assets/js.js"></script> <script src="/githubissues/assets/markdown.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/highlight.min.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/languages/go.min.js"></script> <script> hljs.highlightAll(); </script> </body> </html>