Can't import in a Docusaurus project - error `can't resolve 'http'`

[AchrafAmil]

Description

My goal is to deploy two docs (OpenAPI & AsyncAPI) together in the same place.

1. I generate a Docusaurus website using steps docusaurus-openapi. The result is at the "Initial commit" of the reproducible minimal example.

2. I add a custom react page as described in their doc and import a random react component in it ('awesome-react-button'), and it works:

3. I import the AsyncAPI React component (exact diff commit) but now it crashes:

Expected result

Import to work and not break everything.

Actual result

Compilation errors

Full errors snippet

``` Module not found: Error: Can't resolve 'http' in '/Users/achraf/exp_docs/docusaurus/my-website/node_modules/@apidevtools/json-schema-ref-parser/lib/resolvers' Did you mean './http'? Requests that should resolve in the current directory need to start with './'. Requests that start with a name are treated as module requests and resolve within module directories (/Users/achraf/exp_docs/docusaurus/my-website/node_modules/@docusaurus/core/node_modules, node_modules, /Users/achraf/exp_docs/docusaurus/my-website/node_modules). If changing the source code is not an option there is also a resolve options called 'preferRelative' which tries to resolve these kind of requests in the current directory too. BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default. This is no longer the case. Verify if you need this module and configure a polyfill for it. If you want to include a polyfill, you need to: - add a fallback 'resolve.fallback: { "http": require.resolve("stream-http") }' - install 'stream-http' If you don't want to include a polyfill, you can use an empty module like this: resolve.fallback: { "http": false } Module not found: Error: Can't resolve 'https' in '/Users/achraf/exp_docs/docusaurus/my-website/node_modules/@apidevtools/json-schema-ref-parser/lib/resolvers' BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default. This is no longer the case. Verify if you need this module and configure a polyfill for it. If you want to include a polyfill, you need to: - add a fallback 'resolve.fallback: { "https": require.resolve("https-browserify") }' - install 'https-browserify' If you don't want to include a polyfill, you can use an empty module like this: resolve.fallback: { "https": false } Module not found: Error: Can't resolve 'stream' in '/Users/achraf/exp_docs/docusaurus/my-website/node_modules/avsc/etc/browser' BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default. This is no longer the case. Verify if you need this module and configure a polyfill for it. If you want to include a polyfill, you need to: - add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }' - install 'stream-browserify' If you don't want to include a polyfill, you can use an empty module like this: resolve.fallback: { "stream": false } Module not found: Error: Can't resolve 'stream' in '/Users/achraf/exp_docs/docusaurus/my-website/node_modules/avsc/lib' BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default. This is no longer the case. Verify if you need this module and configure a polyfill for it. If you want to include a polyfill, you need to: - add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }' - install 'stream-browserify' If you don't want to include a polyfill, you can use an empty module like this: resolve.fallback: { "stream": false } Module not found: Error: Can't resolve 'zlib' in '/Users/achraf/exp_docs/docusaurus/my-website/node_modules/avsc/lib' BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default. This is no longer the case. Verify if you need this module and configure a polyfill for it. If you want to include a polyfill, you need to: - add a fallback 'resolve.fallback: { "zlib": require.resolve("browserify-zlib") }' - install 'browserify-zlib' If you don't want to include a polyfill, you can use an empty module like this: resolve.fallback: { "zlib": false } Module not found: Error: Can't resolve 'stream' in '/Users/achraf/exp_docs/docusaurus/my-website/node_modules/avsc/lib' BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default. This is no longer the case. Verify if you need this module and configure a polyfill for it. If you want to include a polyfill, you need to: - add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }' - install 'stream-browserify' If you don't want to include a polyfill, you can use an empty module like this: resolve.fallback: { "stream": false } client (webpack 5.88.0) compiled with 6 errors ```

Steps to reproduce

Checkout before/after this commit in the reproducible: https://github.com/AchrafAmil/DocusaurusAsyncAPI/commit/7370bc343020ff4a6632fa72edc58b9474e63bab

Troubleshooting

Web dev is really not my comfort zone so I might be doing/saying something wrong, but it seems like the AsyncAPI React component is using a node.js dependency that shouldn't be there (?).

[github-actions[bot]]

Welcome to AsyncAPI. Thanks a lot for reporting your first issue. Please check out our contributors guide and the instructions about a basic recommended setup useful for opening a pull request.
Keep in mind there are also other channels you can use to interact with AsyncAPI community. For more details check out this issue.

[fmvilas]

Yup, it looks like Docusaurus is a pure client-side app. AsyncAPI React component is not purely client-side but we're publishing a client-side version of it that I guess that's what you should be using.

Can you try using the next version? https://github.com/asyncapi/asyncapi-react/tree/next. You'll find docs there on how to use it even though the rewrite is not finished yet it's pretty close to be done.

cc @magicmatatjahu

[derberg]

@AchrafAmil please check with latest release candidate -> v1.0.0-next.48

@fmvilas AsyncAPI React component is not purely client-side - react component is definitely client side, not sure what you mean

[AchrafAmil]

Just tried v1.0.0-next.48 and I still get the same error.

[fmvilas]

It is not because it depends on the AsyncAPI parser that access the Node.js fs package. Last time I checked, we were using browserify to make it browser-compatible. Not sure it's still the same.

[derberg]

asyncapi-react still uses old parser. And yes, in old parser there are some file system operations, and also there is node-fetch used when you do parseFromUrl but that is what browserify/webpack/younameit 🤷 the fact that you use them make your package also client-side, because I use package, and don't care about source code. But yeah, just my opinion 😛

@AchrafAmil the reason is https://github.com/APIDevTools/json-schema-ref-parser/blob/v9.0.6/lib/resolvers/http.js#L3 and after short investigation I think it is because of the node version that you are using 🤔 so http and also https are native node packages. Last time in their docs they say to use it like require('http') is Node 14, the new ones like 16 and up say require('node:http'). So I think you are using Node higher than 14 and it complains about require('http') because it by default things it is a broken local code file reference.

Try to use Node 14, maybe that helps

[AchrafAmil]

@derberg I can't force Node 14 as it's not supported by facebook/docusaurus

@docusaurus/core@2.4.1: The engine "node" is incompatible with this module. Expected version ">=16.14". Got "14.21.3"

[AchrafAmil]

and if I "ignore engines" I get this (very likely some new syntax that node 14 can't get)

file:///Users/achraf/exp_docs/docus2/myws/node_modules/@docusaurus/core/bin/docusaurus.mjs:30
process.env.BABEL_ENV ??= 'development';
                      ^^^

SyntaxError: Unexpected token '??='
    at Loader.moduleStrategy (internal/modules/esm/translators.js:149:18)
    at async link (internal/modules/esm/module_job.js:67:21)

[derberg]

oh not cool 😞

you have 2 options:

• wait once we migrate to use in react the new v2 parser that uses different package that do not use http dependency. I cannot commit to any dates. Unless @fmvilas planned something already
• you can try workaround this problem with https://www.npmjs.com/package/module-alias if you are in rush. Below you see chatgpt hint that sounds like it could work

[fmvilas]

Unless @fmvilas planned something already

Not yet but my idea is to give it a bump around September.

[echosergio]

@AchrafAmil another workaround, until a new version of the AsyncApiComponent is released (next branch), is by using the Standalone Bundle loaded in the useEffect hook of a custom component:

function AsyncApiBrowserStandalone({ schema, config }: ContainerProps) {
  useEffect(() => {
    const AsyncApiStandalone = require('@asyncapi/react-component/browser/standalone');

    AsyncApiStandalone.render(
      {
        schema: schema,
        config: config,
      },
      document.getElementById('asyncapi'),
    );

  }, [schema, config]);

  return <div id="asyncapi">Loading...</div>;
}

You may also load the standalone script in the scripts section of the docusaurus.config.js, if so, remeber to also manually load the stylesheet.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity :sleeping:

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience :heart:

[kennethaasan]

still an issue

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity :sleeping:

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience :heart:

[MartinMuzatko]

Still an issue. Even have the same problem in other client-side only projects like vite. However @echosergio solution works fine. I'd just change the document.getElementById solution with a ref to allow multiple renders of the same component.

import { useEffect, useRef } from "react";
import { AsyncApiProps } from "@asyncapi/react-component";
import "@asyncapi/react-component/styles/default.min.css";

// This wrapper is needed to load the standalone version of asyncapi
// since the original version relies on server-side rendering
export function AsyncApi({ schema, config }: AsyncApiProps) {
  const ref = useRef<HTMLDivElement | null>(null);
  useEffect(() => {
    const AsyncApiStandalone = require("@asyncapi/react-component/browser/standalone");
    AsyncApiStandalone.render(
      {
        schema,
        config,
      },
      ref.current
    );
  }, [schema, config]);

  return (
    <div ref={ref} id="asyncapi">
      Loading...
    </div>
  );
}

[AchrafAmil]

FWIW, I went for the (pretty savage but quick) workaround of simply putting the fat single-file html+css+javascript in a React component. It gives the following results: https://docs.nabla.com/server/copilot-listen-ws