Selection and Keyboard Navigation of Media Blocks

[thesunny]

This is a problem I've been thinking about for a week and I thought I'd start discussion on it.

The problem is how selection and keyboard navigation works with Media blocks.

I'm also focusing on the keyboard not the mouse.

There are two ways media types like images work in other editors: (1) as inline elements (e.g. in Google Drive) and (2) as blocks (e.g. in Medium). I'm specifically targeting their use as blocks.

Inline Media Behavior

For completeness of understanding though I'll go over Inline media first. The typical way to select inline media elements is that the caret moves around them. If there are two images, a caret moving from left to right would look like the following where X is the image:

|XX X|X XX|

In order to select an image, one could hold down the shift key. For example, hold shift then key right one to select the first image:

shift + right arrow results in

|X|X

Current Media Block Navigation Behavior

When the media types are implemented as blocks in DraftJS, we typically make the blocks contentEditable={false}. When we mouse down from above 2 media blocks, the selection skips the media blocks.

|
X
X

Key down arrow results in:

X
X
|

Media Block Navigation Behavior I'd Like to Support

What I'd like to have the DraftJS editor be able to support, is to have the key down arrow to instead select the first media block. This behavior is that adopted by the Medium editor.

|X| X

Another key down arrow results in

X |X|

Something like this is important. Consider 10 media blocks stacked on top of each other. A key down arrow would skip more than an entire screenful. This is jarring.

Alternate Block Navigation Behavior (Not Recommended Personally)

Some other options would allow the cursor to be in position around the media block but it feels a little less straightforward and I don't see any utility above the implementation above. For completeness though, a user could key right to get all these positions:

X

key right

|X X

key right

X| X

key right

X |X

And so forth.

Discussion of Solution

Note: I'm not advocating for selection of media blocks to necessarily be baked into DraftJS although this may not be a bad idea. I'm hoping for a solution that could work for this common use case.

The biggest issue is the skipping of all contentEditable={false} blocks.

One thing I've been thinking about is to leave the root media block as contentEditable={true} and only have a contentEditable={false} div inside. When the cursor moves into that contentEditable position, we recognize the caret is within the media block, and we can restyle the block with an outline or with handles or whatever we choose. In this scenario, we would also have to figure out a way to make the blinking caret be invisible, perhaps with some CSS. Not sure if this is even possible though.

However, my short attempt at this was unsuccessful. The cursor would go into the media block, and when we type a character, it shows up within the media block. However, it's clear that DraftJS is in an inconsistent DOM state. If I save then load the blocks (using the convert to/from RAW, the characters typed in the media block are in different positions after the load).

I am pleased that the delete and backspace around blocks seems to work correctly already; however, the selection skipping is a problem I think that would be common to anybody that uses the media block.

Has anybody else solved this issue? And if so how?

[thesunny]

I haven't had time to try this but in case anyone wants to give it a shot, the newer versions of DraftJS support keyBindingFn. Basically it triggers before a key is typed.

Theoretically, we should be able to do an event.preventDefault() to stop the text from being typed when the selection is within a Media Block.

[davidbyttow]

Thank you for typing this up, @thesunny, this is an almost exact articulation of what I'm thinking about too. I'm surprised that it hasn't gotten more attention/comments though.

Aside: It seems Facebook Notes handles this by not allowing you to "select" an image, but instead just manipulate it directly by selecting options on-hover (which is a fairly reasonable alternative to Medium's style).

Your initial solution, paired with the prevention of text typing/pasting/etc sounds workable. And I do believe there is a generalized use case, where essentially you want an entire block to be selectable, either by selecting it with the mouse or by moving the cursor "into" it (e.g., Medium-style). And your suggestion may be the path to reaching that generalized solution.

I started considering another approach, where you can override the cursor movement behavior, prevent it, and "select" the block that it was attempting to navigate to. So, if you press "down" toward a Media block, it would be intercepted, focus removed and the component that renders the block would be put into a selected mode. That means, of course, that the component would then need to handle subsequent movement commands, refocus the editor, and move the cursor down (or onto the next Media block).

I think the above is more like what Medium does, because the cursor focus gets placed into the caption of the image (you can sort of tell by copy/pasting something while the image is selected).

However, I feel like your proposed solution may be more workable and contained.

Has anyone tried this?

[hellendag]

This is an issue that plagued me while working on Notes, as I couldn't quite get things to a place I was really happy with. I'd have something that sort of worked, then fail certain arrow behavior cases or not do what I wanted when selecting across multiple blocks. I never felt like I had something shippable.

In order to keep the product moving, I came up with the current hover menus and punted on trying to solve whole-block selection. No doubt a solution exists, I just didn't have the time to sink into it.

I am very much in favor of either identifying a reliable reusable pattern to handle selection behavior for this use case, or possibly encapsulating it within a core wrapper component.

@tasti and I have chatted a bit about this. With a standardized AtomicBlock component, encapsulation may be possible.

[oyeanuj]

@tasti @hellendag Any updates here?

@thesunny @guitardave24 Did any of you end up solving this separately?

Another usecase to consider here is when the editor has multiple images and wanting to select one or more image to perform an operation (like layout, etc).

[ianstormtaylor]

For what it's worth, I've solved this exact kinda of use case (not all of the way) with the following plugin: https://gist.github.com/ianstormtaylor/a0ed19db8b0caed7a48e28235b672446

You have to do some weird things to get the cursor to be inside the block but for it to not be visible.

Not sure if it's the end solution I'd go with. I was just thinking the other day that it might be better to actually leave the selection across the entire set of children inside the block but using ::selection styles to make it transparent. Either that, or cmd-c/x needs to be handled separately as well...

Definitely tricky, but +1 having a solution to this. Would be huge.

[noahlemen]

@tobiasandersen -- based on https://github.com/facebook/draft-js/pull/971#issuecomment-276219059, it sounds like you may have encountered this issue as well. If so, I'm curious how you handle selection of adjacent atomic blocks (as mentioned in the "Current Media Block Navigation Behavior" in the original post here)?

[tobiasandersen]

@kedromelon Yes, we've created a custom block that mimics the behavior label as Media Block Navigation Behavior I'd Like to Support (in the original post). So we're basically doing what Medium does.

Our solution is based on the ideas shared in the gist above. It's far from perfect, but I'm pretty happy with what we've got so far. I'm thinking about breaking it out and release openly somewhere, since there seems to be quite a lot of people interested in it. I'm not sure what the best way would be though. Maybe as a plugin for the draft-js-plugins project?

[noahlemen]

@tobiasandersen I think that'd certainly be a useful resource to make available! Seems like a common behavior to implement. I haven't used draft-js-plugins personally, so I can't attest to whether or not it would fit in there, but it seems like it might.

Would love to contribute to a plugin like that. Let me know if you end up making it available + want any help on it.

[tobiasandersen]

@kedromelon Yeah that'd be great! I'll keep you posted.

[optimatex]

i'm interesting in same. It's the most valuable feature that i need for draft.js