Spike to investigate using lexical for our editor

[burtonator]

I think we should investigate using Lexical directly and just bypass MDXEditor.

@ForestMars

Here are the main things I need to investigate in the next spike:

• [x] 1h: Create a new / basic repo with create-react-app and use lexical directly to experiment with the devx. This should import the packages, initialize lexical, and let me type into the control and render it on the page.
• [x] 1h: Make sure we can type markdown in the keyboard and that it's converted to bold, italics, etc.
• [x] 1h: Verify that I can make it work with the basic functionality we need and without a store. Specifically, getting the markdown when the user clicks a button.
• [x] 3h: how do I get the basic functionality of lexical to work with their default markdown functionality / plugin.
• [x] 2h: selection: I think some buttons could/should only work when the selection is active. I need to check this and verify that it will work properly and figure out which options are only active during selection (if any)
• [x] 1h: Can I invoke all the main toolbar actions we want (bold, italic, undo, redo)
• [x] 1h: How do we make tables and links work that might require a state machine and are more complex?
• [x] 3h: How do we implement a plugin like codemirror for source code highlighting (MDXEditor does it)
• [x] 2h: what is MDXEditor doing with the mdast extensions and do we need them?
• [x] 1h: ... same thing with micromark
• [x] 2h: ... investigate building plugins with lexical and read more about the internals.

... 14h total but it will probably be less than that. I'm mostly worried about the markdown functionality and I want to really investigate this to make sure we're not missing something.

The other issue is that there's no real plugin support for codemirror or mermaid.js or any other GFM extensions so we either:

1. Just accept that we NEVER implement anything like this.
2. Investigate it now but I think it will add significantly more time to the spike
3. Just risk it and move on. I should note that MDXEditor implements codemirror for source code highlighting so it's definitely possible.

There's also the issue with Github Flavored Markdown (GFM) and other extensions that we might need to support.

What I'm worried about with this issue is that unless we really support all these extensions in our editor that we might paint ourselves into a corner in the future because we standardize on Lexical. I think what MDXEditor does is implement some of this via micromark and mdast but I need to investigate.

[burtonator]

Quick status update. I have the first step done. I have a repo where I can work with lexical.

The next step is to figure out how to make it do BASIC markdown import and export. Lexical has support for this natively I think.

I think we could break this out into two spikes. This first is just lexical with BASIC markdown support. Then I can look at more advance exports later.

[burtonator]

Another thing I realized is that if we go down this path to support Lexical directly, I think we can implement @ mentions where we popup and autocomplete widget and / (slash commands) where we can supports commands ALA Notion or github.

I think this could give us like 90% of an advanced editor but internally I'd prefer it not just be one contenteditable.

I'm still working through this though and trying to understand Lexical's model.

[burtonator]

Other things we have to test for v1 (basic editor)

• [ ] initialize with existing markdown
• [ ] ability to resize the content editable region
• [x] Headings
• [x] inline code
• [x] pasting rich text and make sure it is properly converted to markdown
• [x] auto linking (type a URL and it becomes an A element)
• [x] Working with lists on the keyboard works and you can exit the list mode
• [x] undo/redo
• [x] tables and figure out if we have to have our own state and framework for handling the UI
• [ ] multiline code
• [ ] make sure full markdown toolbar features work with dispatchCommand (like MDXEditor)
• [x] pasting images (data URL for now and upload to cloud in prod)
• [x] dragging and dropping images
• [ ] complete the toolbar for all the main options (headings, list items, etc)
• [x] List items (numbered and list)

Issues:

• underline doesn't work for markdown right now and we will need to either disable it or convert it to an HTML ul element. Here's a good thread on the subject: https://stackoverflow.com/questions/3003476/get-underlined-text-with-markdown
• We will have to test on Safari + Firefox
• Some HTML editors will show the toolbar items disabled or enabled based on the cursor state. I think we can do that but I think it's overkill

Notes:

• tables are going to be complicated but still straight forward. We have to implement our own TableNode and both Lexical and MDXEditor have implementations of these. MDXEditor is better though.
• pasting images will have to be implemented via dispatchCommand(INSERT_IMAGE) and then we have to have a react component to listen to drag and drop or paste, then convert the pasted data to a data URL or upload them to the cloud. We'll need a UI to show that something is pending though and maybe an image placeholder during that time...
• There is a "clickable link" plugin but the one we have now doesn't have a UI

[burtonator]

We also need to discuss which features/plugins from Lexical and MDEditor we should adopt and we also need to make sure they are supported in markdown.

Lexical Playground

Link Popup

Code blocks

Draggable blocks ALA notion

Latex / Math support

OEmbed and Youtube Embed

We could consider adding support for oembed but this would be more work in the future.

https://oembed.com/

@ mentions

hashtags

hover toolbar on desktop

[burtonator]

UPDATE: This is how we would build our own editor. I have a potential solution for how to use MDXEditor directly. See below.

I'm pretty much completed the research on this spike. I still want to read a bit more about the internals of lexical to help make a better decision but here is where we stand so far.

I think there are two paths we can go moving forward.

Plan A: Fork MDXEditor and port it to our UI/UX framework.

Basically we would fork the MDXEditor and customize it internally to do what we want it to do but just deal with the idiosyncrasies of that framework. We'd have to port it from RadixUI to our own UI and then figure out what to do with their store.

I don't understand how their store works and they use a proprietary store called GuruX.

pros:

• it would be starting with something that we know works and I think in the short term it would be faster but in the long term cost us a lot more work. It would probably take me a week of work to clean it up for our use and then we'd pay for it in the long term after that.

cons:

• We're going to be anchored to the API decisions and technical architecture they have made and it's going to mean we have a sub-par editor in the long term.

Plan B: Implement our own editor on top of Lexical

I would prefer this plan. I think the main challenges here involved:

• Tables: This actually turns out the be the hardest part which I didn't originally anticipate. Tables are a plugin so require a custom react UI. I think I can take the table code from MDXEditor and just use it in our editor. It's under an MIT license so that's fine.

• Plugins: I don't fully understand the plugin API right now but implementing tables would give me a good feel for how hard it's going to be. We can really implement some amazing functionality here including embeds, notion-style drag and drop, autocompletion, custom toolbars, etc.

pros:

• We'd have an amazing and first-class editor without much baggage and the ability to extend it over the long term
• Since it's based on Lexical I think the bugs due to contenteditable would be minimal and that it would be maintainable
• We would be able to handle advanced used cases like custom toolbars, mobile toolbars, source code editing, embeds, etc.
• Fully WYSIWYG
• I think going with plan B would take the same amount of time that plan A would take but we'd end up with something that's easier to support and with more features!

cons:

• I need probably 1-2 days to port over the table code from MDXEditor and to understand how they use their store. It's really just 1-2 files though without any other files being used or weird dependencies. The store and Radix-UI are the main obstacles.
• It would be my our first lexical plugin and I haven't developed one before but they seem straight forward.
• We'd have to test it and support it and if there are many regressions we'd have to use Karma to test browser interactions. I think though that Meta/Facebook would prevent any obstacles like this from hitting the code though.

[burtonator]

There are other issues here including:

• pasting of content: https://github.com/hicommonwealth/commonwealth/issues/7510
  • making sure the algorithm for converting markdown to HTML is compatible with our editor

[burtonator]

We can fit 9-10 buttons on mobile...

[burtonator]

I think we might be able to use MDXEditor for now if:

• we can use <= 10 buttons on mobile
• We can keep the formatting buttons up at all time and don't need to swap it on and off
• We're only on light mode.

I think we only need to following buttons on mobile:

• bold
• italics
• bulleted items
• numbered items
• link
• H1
• H2
• image