Sidebar generation will run `Globby` in the `node_modules` folder

rle-mino commented 3 months ago

Have you read the Contributing Guidelines on issues?

[X] I have read the Contributing Guidelines on issues.

Prerequisites

[X] I'm using the latest version of Docusaurus.
[X] I have tried the npm run clear or yarn clear command.
[X] I have tried rm -rf node_modules yarn.lock package-lock.json and re-installing packages.
[ ] I have tried creating a repro with https://new.docusaurus.io.
[X] I have read the console error message carefully (if applicable).

Description

Hi,

I'm using Docusaurus for a monorepo where every package can have markdown files. When adding the plugin-content-docs to a specific package, the start command gets stuck:

[INFO] Starting the development server...

without more information.

I looked into the code and found that it was an issue during sidebar generation, the entire process gets blocked running this line of code: https://github.com/facebook/docusaurus/blob/main/packages/docusaurus-plugin-content-docs/src/sidebars/index.ts#L44

It looks like Globby runs on the node_modules folder. When adding ignore: ['**/node_modules'] in the options, Docusaurus starts perfectly!

Is there any reason to look for the _category_.* config files in the node_modules folder?

Reproducible demo

No response

Steps to reproduce

Have big dependencies in your package.json
Run docusaurus start

Expected behavior

Should starts docusaurus

Actual behavior

gets stuck on

[INFO] Starting the development server...

Your environment

Docusaurus version used: 3.4.0
Environment name and version (e.g. Chrome 89, Node.js 16.4): Node.js 20.15.0
Operating system and version (e.g. Ubuntu 20.04.2 LTS): MacOS 14.2.1

Self-service

[ ] I'd be willing to fix this bug myself.

Josh-Cena commented 3 months ago

Do you have node_modules in your content path? I don't think that's supposed to happen. Otherwise by default you would also have any Markdown files there automatically becoming doc pages, or other bad things happening. Please give us an idea of how your project is set up, at least.

rle-mino commented 3 months ago

Do you have node_modules in your content path? I don't think that's supposed to happen. Otherwise by default you would also have any Markdown files there automatically becoming doc pages, or other bad things happening. Please give us an idea of how your project is set up, at least.

Yes, I do have a node_modules folder in my content path. Here's the content folder's structure:

monorepo-project
- node_modules/
- package.json
- packages/
- - package-a/ # Content directory
- - - node_modules/ # <- Analysed by Globby (runs for more than 5 minutes)
- - - src/ # <- Has .md and .mdx files
- - - docs/ # <- has .md and .mdx files
- - - README.md # <- Should be in the documentation
- - - package.json
- - package-b/
- - package-c/

So I wan't to run plugin-content-docs for the entire package-a folder (to avoid complex sidebar configuration).

slorber commented 2 months ago

Can you create a realistic repro docusaurus.new of your monorepo setup?

It would help to see the full picture, including config files, and be able to run it easily.

Josh-Cena commented 2 months ago

I wouldn't recommend you setting contentPath to be the entire package-a. That would definitely include node_modules. Instead, you should have a dummy doc in your docs folder that contains this:

import Readme from "../README.md";

<Readme />

This is what we do for our own website: https://github.com/facebook/docusaurus/blob/main/website/community/3-contributing.mdx

facebook / docusaurus