search

payloadcms / payload

Payload is the open-source, fullstack Next.js framework, giving you instant backend superpowers. Get a full TypeScript backend and admin panel instantly. Use Payload as a headless CMS or for building powerful applications.
https://payloadcms.com
MIT License
24.8k stars 1.58k forks source link

Docs for HTML -> Lexical #7440

Open ainsleyclark opened 3 months ago

ainsleyclark commented 3 months ago

Link to reproduction

No response

Payload Version

3.0.0-beta.67

Node Version

v18.17.1

Next.js Version

15.0.0-canary.53

Describe the Bug

The documentation provided on the payload/richtext-lexical page isn't 100% clear for converting HTML -> Lexical.

Originally reported on Discord, I'm trying to write a script that takes HTML and converts it to Lexical JSON.

import { createHeadlessEditor } from '@lexical/headless';
import { $generateNodesFromDOM, $generateHtmlFromNodes } from '@lexical/html';
import { $getRoot, $getSelection } from 'lexical';
import { JSDOM } from 'jsdom';
import type { SerializedEditorState } from 'lexical';
import { getEnabledNodes, sanitizeServerEditorConfig, defaultEditorConfig } from '@payloadcms/richtext-lexical'

export const htmlToLexical = (html: string): SerializedEditorState => {
    const editor = createHeadlessEditor({
        nodes: [],
        onError: (error) => { console.error(error); },
    });

    editor.update(
        () => {
            // In a headless environment you can use a package such as JSDom to parse the HTML string.
            const dom = new JSDOM(`<!DOCTYPE html><body>${html}</body>`);

            // Once you have the DOM instance it's easy to generate LexicalNodes.
            const nodes = $generateNodesFromDOM(editor, dom.window.document);

            // Select the root
            $getRoot().select();

            // Insert them at a selection.
            const selection = $getSelection();

            console.log("Generated nodes: ", nodes);

            if (selection) selection.insertNodes(nodes);
        },
        { discrete: true },
    );

    return editor.getEditorState().toJSON();
};

I call it like so:

const scratch = () => {
    const html = htmlToLexical(
        '<h2>Online English Lessons</h2><p>Are you worried about making mistakes</p>',
    )
    console.log(JSON.stringify(html, null, 2))
}

Unfortunately, it doesn't generate headings and I'm not sure why?

"root": {
    "children": [
      {
        "children": [
          {
            "detail": 0,
            "format": 0,
            "mode": "normal",
            "style": "",
            "text": "Online English Lessons",
            "type": "text", // Should be heading?
            "version": 1
          }
        ],
        "direction": null,
        "format": "",
        "indent": 0,
        "type": "paragraph",
        "version": 1,
        "textFormat": 0
      },
    ],
    "direction": null,
    "format": "",
    "indent": 0,
    "type": "root",
    "version": 1
  }
}

I have tried to use getEnabledNodes but it's throwing an error:

const editor = createHeadlessEditor({
    nodes: getEnabledNodes({
        //@ts-ignore
        editorConfig: sanitizeServerEditorConfig(defaultEditorConfig, {}),
    }),
    onError: (error) => {
        console.error(error);
    },
});

// Results in:
// TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".css" for /Users/Ainsley/Desktop/ainsley.dev/WebKit/web-kit/node_modules/.pnpm/react-image-crop@10.1.8_react@18.3.1/node_modules/react-image-crop/dist/ReactCrop.css

It would be great to get a full example of how to convert HTML to Lexical & visa versa as it's not 100% clear how to setup the headlessEditor.

Thanks in advance.

Reproduction Steps

Run script above.

Adapters and Plugins

No response

chrisvanmook commented 2 months ago

@ainsleyclark I'm using the following, maybe this helps:

import { createHeadlessEditor } from '@lexical/headless';
import { $generateNodesFromDOM } from '@lexical/html';
import {
  defaultEditorConfig,
  getEnabledNodes,
  sanitizeServerEditorConfig,
} from '@payloadcms/richtext-lexical';
import { JSDOM } from 'jsdom';
import { $getRoot, $getSelection } from 'lexical';
import type { SanitizedConfig } from 'payload';

export const HTMLToLexical = async (
  payloadConfig: SanitizedConfig,
  htmlInput: string,
) => {
  const editorConfig = await sanitizeServerEditorConfig(
    defaultEditorConfig,
    payloadConfig,
  );

  const headlessEditor = createHeadlessEditor({
    nodes: getEnabledNodes({
      editorConfig,
    }),
  });

  headlessEditor.update(
    () => {
      const dom = new JSDOM(htmlInput);

      const nodes = $generateNodesFromDOM(headlessEditor, dom.window.document);

      $getRoot().select();

      const selection = $getSelection();
      selection?.insertNodes(nodes);
    },
    { discrete: true },
  );

  return headlessEditor.getEditorState().toJSON();
};
ainsleyclark commented 2 months ago

Thanks @chrisvanmook that's helpful. May I ask what version you're using? I'm getting problems with a CSS file import.

chrisvanmook commented 2 months ago

Thanks @chrisvanmook that's helpful. May I ask what version you're using? I'm getting problems with a CSS file import.

That's the thing, you have to match the versions payload is using in their code. I hope payload offers a more solid solution for this in the future, for now I just manually check if they match. These are the ones I'm using now:

    "jsdom": "^24.1.1",
    "jsonschema": "^1.4.1",
    "lexical": "^0.16.1",
    "next": "15.0.0-canary.77",
    "payload": "3.0.0-beta.68",
    "@lexical/headless": "^0.16.1",
    "@lexical/html": "^0.16.1",
    "react": "19.0.0-rc-fb9a90fa48-20240614",
    "react-dom": "19.0.0-rc-fb9a90fa48-20240614",
ainsleyclark commented 2 months ago

Thanks for your help @chrisvanmook but I'm still getting:

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".css" for /node_modules/.pnpm/react-image-crop@10.1.8_react@19.0.0-rc-fb9a90fa48-20240614/node_modules/react-image-crop/dist/ReactCrop.css
import { createHeadlessEditor } from '@lexical/headless';
import { $generateNodesFromDOM, $generateHtmlFromNodes } from '@lexical/html';
import { $getRoot, $getSelection, type LexicalEditor } from 'lexical';
import { JSDOM } from 'jsdom';
import type { SerializedEditorState } from 'lexical';
import { getPayload, buildConfig } from 'payload';
import { importWithoutClientFiles } from 'payload/node'
import { sqliteAdapter } from '@payloadcms/db-sqlite';
import {
    defaultEditorConfig,
    getEnabledNodes,
    lexicalEditor,
    sanitizeServerEditorConfig,
} from '@payloadcms/richtext-lexical';

const loadEditor = async (): Promise<LexicalEditor> => {
    const config = {
        secret: 'testing',
        editor: lexicalEditor({
            admin: {
                hideGutter: false,
            },
        }),
        db: sqliteAdapter({
            client: {
                url: 'file:./local.db',
            },
        }),
    };

    const instance = await getPayload({
        config: buildConfig(config),
    });

    const editorConfig = await sanitizeServerEditorConfig(defaultEditorConfig, instance.config);

    return createHeadlessEditor({
        nodes: getEnabledNodes({
            editorConfig,
        }),
    });
};

/**
 * Converts an HTML string to a Lexical editor state.
 *
 * @param {string} html - The HTML string to convert.
 * @returns {SerializedEditorState} The serialized editor state.
 */
export const htmlToLexical = (html: string): SerializedEditorState => {
    let state = {};

    loadEditor().then((editor) => {
        editor.update(
            () => {
                // In a headless environment you can use a package such as JSDom to parse the HTML string.
                const dom = new JSDOM(`<!DOCTYPE html><body>${html}</body>`);

                // Once you have the DOM instance it's easy to generate LexicalNodes.
                const nodes = $generateNodesFromDOM(editor, dom.window.document);

                // Select the root
                $getRoot().select();

                // Insert them at a selection.
                const selection = $getSelection();

                if (selection) selection.insertNodes(nodes);
            },
            { discrete: true },
        );

        state = editor.getEditorState().toJSON();
    });

    return state as SerializedEditorState;
};
chrisvanmook commented 2 months ago

Thanks for your help @chrisvanmook but I'm still getting:

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".css" for /node_modules/.pnpm/react-image-crop@10.1.8_react@19.0.0-rc-fb9a90fa48-20240614/node_modules/react-image-crop/dist/ReactCrop.css

Hmm can't reproduce this, unfortunately. Seems like a dependency of @payloadcms/ui. Not sure if it's related to this specific code.

foxted commented 2 months ago

It's happening on my end as well, here are my dependencies:

@payloadcms/plugin-cloud 3.0.0-beta.82       @payloadcms/ui 3.0.0-beta.82                 graphql 16.9.0                               react 19.0.0-rc-fb9a90fa48-20240614
@payloadcms/db-postgres 3.0.0-beta.82        @payloadcms/richtext-lexical 3.0.0-beta.82   change-case 5.4.4                            next 15.0.0-canary.53                        react-dom 19.0.0-rc-fb9a90fa48-20240614
@payloadcms/next 3.0.0-beta.82               @payloadcms/storage-s3 3.0.0-beta.82         cross-env 7.0.3                              payload 3.0.0-beta.82                        sharp 0.32.6

devDependencies:
@swc/core 1.7.10
@types/node 20.14.10
eslint 8.57.0
eslint-config-next 15.0.0-canary.53
types-react 19.0.0-rc.0
types-react-dom 19.0.0-rc.0
typescript 5.5.4
AlessioGr commented 2 months ago

@foxted this should be fixed in beta.82. Can you provide the full error? And did you finish the migration of your payload config to component paths?

foxted commented 2 months ago

This is the full error: 

> cms-marketing@1.0.0 generate:types /Users/valentinprugnaud/Sites/Faro/faroseparations.com/cms/marketing
> payload generate:types

node:internal/process/promises:391
    triggerUncaughtException(err, true /* fromPromise */);
    ^
TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".css" for /Users/valentinprugnaud/Sites/Faro/faroseparations.com/node_modules/.pnpm/react-image-crop@10.1.8_react@19.0.0-rc-fb9a90fa48-20240614/node_modules/react-image-crop/dist/ReactCrop.css
    at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:160:9)
    at defaultGetFormat (node:internal/modules/esm/get_format:203:36)
    at defaultLoad (node:internal/modules/esm/load:143:22)
    at async nextLoad (node:internal/modules/esm/hooks:866:22)
    at async load (file:///Users/valentinprugnaud/Sites/Faro/faroseparations.com/node_modules/.pnpm/tsx@4.17.0/node_modules/tsx/dist/esm/index.mjs?1723772901192:2:1762)
    at async nextLoad (node:internal/modules/esm/hooks:866:22)
    at async Hooks.load (node:internal/modules/esm/hooks:449:20)
    at async handleMessage (node:internal/modules/esm/worker:196:18) {
  code: 'ERR_UNKNOWN_FILE_EXTENSION'
}

Node.js v20.16.0
 ELIFECYCLE  Command failed with exit code 1.

I was on beta.77 prior to that update on local, I am not sure what you mean by "migration of my payload config to component paths"? Is the payload.config.ts different now?

AlessioGr commented 2 months ago

I was on beta.77 prior to that update on local, I am not sure what you mean by "migration of my payload config to component paths"? Is the payload.config.ts different now?

Oh yea, this error might not even be coming from lexical. Check out the migration guide here: https://github.com/payloadcms/payload/releases/tag/v3.0.0-beta.79

foxted commented 2 months ago

Thanks, somehow I missed these steps! However, the issue still occurs, but only on one of the two instances of Payload that I'm running, I'll try to pin-point the difference.