search

mdx-js / mdx

Markdown for the component era
https://mdxjs.com
MIT License
17.77k stars 1.14k forks source link

docs: update Parcel MDX plugin for MDX 3 #2523

Closed TechQuery closed 1 month ago

TechQuery commented 3 months ago

Initial checklist

Description of changes

Parcel official MDX plugin is only support MDX 1, so I publish a new plugin for MDX 3.

vercel[bot] commented 3 months ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
mdx ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 6, 2024 8:04pm
codecov-commenter commented 3 months ago

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (908ff45) to head (f8cd910). Report is 45 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2523 +/- ## ========================================= Coverage 100.00% 100.00% ========================================= Files 23 23 Lines 2693 2712 +19 Branches 2 2 ========================================= + Hits 2693 2712 +19 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

ChristianMurphy commented 3 months ago

Thanks for sharing @TechQuery!

General feedback on what gets included in the plugins/guides list:

Some criteria to include packages in this list.

  • Some documentation with at least some instructions on how to use the package.
  • A CI job to run tests.
  • A prepack script and CI job to build the types.
  • The package should be published to npm.

source: https://github.com/syntax-tree/hast/pull/24#issuecomment-2015384111

It looks like there aren't any types or tests included in the repository. In addition:

TechQuery commented 3 months ago

Thanks for quick reviewing @ChristianMurphy !

It looks like there aren't any types or tests included in the repository.

🚧This is my first Parcel plugin, I'm learning the best practice of developing Parcel plugins with ESM & TS:

  • The repository does not include a LICENSE file, it should have a license. I'd recommend MIT.
  • It looks like the repository uses a mix of npm and pnpm for dependencies and scripts. I'd recommend using only one. I'd recommend npm.

This package is designed for NPM compatibility, andI use PNPM for developing, so it's no mix-using for users.

This plugin has the same goal of Parcel - Zero Configuration, so users can write Markdown files as what they do on GitHub immediately.

  • The plugin does not allow users to specify their own remark and rehype plugins to include, I'd recommend adding this

These lines just aim to find JSX configuration from all kinds of possible files without extra configuration , which recommends TypeScript and is compatible with Babel, this plugin doesn't control the compiler.

ChristianMurphy commented 3 months ago

As an individual coder, I prefer GPLs.

I appreciate that, I don't think unified has a strict policy on license type for adoption (cc @wooorm @remcohaszing) But for wider adoption, I generally recommend licenses that are compatible with Apache 2.0 https://www.apache.org/legal/resolved.html The GPL and LGPL licenses exclude a lot of projects and organizations from adopting.

This package is designed for NPM compatibility, and I use PNPM for developing, so it's no mix-using for users

I'd still recommend building, testing, and publishing with one. They layout packages different and have different resolutions for peer and transitive dependencies. Mixing the two during development could lead to unexpected errors.

Because MDX 3 has no official configuration file, so I'm thinking about the configuration schema of my own plugin. I'm glad to have issue or PR submitted for this feature, and hoping a simple design to keep Parcel easy.

The plugins are remark and rehype based, you could leverage those files, similar to how eslint-mdx does https://github.com/mdx-js/eslint-mdx?tab=readme-ov-file#mdxremark

I use JSDoc & TS to check JS codes internally, but users won't import it in their codes, so .d.ts isn't required.

Even if they aren't published I'd recommend running tsc to ensure they are valid.

I plan to https://github.com/EasyWebApp/Parcel-transformer-MDX/issues/1 to make test scripts easier.

Sounds good

TechQuery commented 3 months ago

The GPL and LGPL licenses exclude a lot of projects and organizations from adopting.

In my opinion, LGPL is designed for libraries, why people keep worrying about it...

I'd still recommend building, testing, and publishing with one. They layout packages different and have different resolutions for peer and transitive dependencies. Mixing the two during development could lead to unexpected errors.

PNPM has more strict Dependency Checking, all packages worked in PNPM also work well in NPM & Yarn.

In the other hand, NPM & Yarn have so bad performance, PNPM is saving my life. But I use it only for local Dependency Management & NPM scripts shorthands, all other things are compatible with NPM.

Even if they aren't published I'd recommend running tsc to ensure they are valid.

When I resolve EasyWebApp/Parcel-transformer-MDX#1, I will rewrite the whole project with TypeScript to replace the quick & dirty version (which is made as the reason of long time waiting for Parcel MDX plugin updating).

wooorm commented 3 months ago

In my opinion, LGPL is designed for libraries, why people keep worrying about it...

Re license: I would very very strongly recommend going with MIT/ISC/similar. That’s what almost all packages in the JS ecosystem go with. Parcel too. And us. Only if you’d have a ton of knowledge about this, and thought this through, would I recommend going against 99% of the npm packages. Also, perhaps you’d be interested in the work on licenses here: https://writing.kemitchell.com/, if you don’t know it already.

I am rather bummed that Parcel hasn’t solved this after a couple years. This reflects poorly on the state of Parcel. Still, this seems like a temporary solution for now, something that will be fixed in the future. Have you considered keeping the original text and link in place, but adding a note linking to your plugin for now, such as this note for Vite?

TechQuery commented 3 months ago

Re license: I would very very strongly recommend going with MIT/ISC/similar. That’s what almost all packages in the JS ecosystem go with. Parcel too. And us. Only if you’d have a ton of knowledge about this, and thought this through, would I recommend going against 99% of the npm packages.

In my mind, GPLs are for powerless individual coders, and others are for powerful companies, universities or foundations.

Until now, no one of open sourcers arounded gives me a clear reason about this, just keeps saying: "Most people use it, so you should use it"... But why? No one knows the logic in it, it doesn't make sense. I will do anything for what makes me believe.

I am rather bummed that Parcel hasn’t solved this after a couple years. This reflects poorly on the state of Parcel. Still, this seems like a temporary solution for now, something that will be fixed in the future.

Yeah, as a long term user, I'm worrying about the team status of Parcel project. But after using webpack, Snowpack, MicroBundle & Vite, only Parcel make me, teammates and students happy on coding with Zero Configuration design.

Have you considered keeping the original text and link in place, but adding a note linking to your plugin for now, such as this note for Vite?

I thought about that, but it'll be confused for users to find a plugin only support MDX 1.x in MDX 3.x document.

ChristianMurphy commented 3 months ago

In my mind, GPLs are for powerless individual coders, and others are for powerful companies, universities or foundations.

Most software is from and for larger organizations and projects. If your intent is for this to be a toy project for individual developers. That is fine, but it doesn't need to be in the main documentation.

Until now, no one of open sourcers arounded gives me a clear reason about this, just keeps saying: "Most people use it, so you should use it"... But why? No one knows the logic in it, it doesn't make sense. I will do anything for what makes me believe.

Concisely, Category X licenses, like LGPL, limit how software can be distributed https://www.apache.org/legal/resolved.html#prohibited The terms get a bit fuzzier when used as a build tool, but it's pretty common for projects to bundle/build on top of build tools, and those would run into the distribution restrictions.

Zero Configuration design

Presets can be created for other build tools and bundlers as well.

confused for users to find a plugin only support MDX 1.x in MDX 3.x document.

I think they'll be more confused if they go out searching for a parcel plugin, see the official one, and don't see it listed in the MDX docs anywhere.

TechQuery commented 3 months ago

Most software is from and for larger organizations and projects. If your intent is for this to be a toy project for individual developers. That is fine, but it doesn't need to be in the main documentation.

This plugin is came from Individual coders, for but not only for individuals.

And most great Open Source projects were Toy projects at the beginning, such as Linux with GPL v2.

Concisely, Category X licenses, like LGPL, limit how software can be distributed https://www.apache.org/legal/resolved.html#prohibited The terms get a bit fuzzier when used as a build tool, but it's pretty common for projects to bundle/build on top of build tools, and those would run into the distribution restrictions.

OK, I will do more research.

Presets can be created for other build tools and bundlers as well.

As I know, only Babel has presets, and other bundlers only have a bunch of configuration fields in their documents, which needs users to set them up manually.

I think they'll be more confused if they go out searching for a parcel plugin, see the official one, and don't see it listed in the MDX docs anywhere.

This plugin is a result of my failed searching about MDX 3.x Parcel plugin, the Parcel official document will tell them that they only support MDX 1.x.

ChristianMurphy commented 3 months ago

most great Open Source projects were Toy projects at the beginning, such as Linux with GPL v2.

Sure, I agree. My point is that the MDX docs focuses on projects that have moved above toy project level.

This plugin is [...] only for individuals.

This concerns me a bit. Many of MDX's adopters are individuals working in/on-behalf of a larger organization. I'm not saying you have to offer corporate enterprise support. But excluding all organization based developers feels a step too far to me (personally, not speaking on behalf of other maintainers).

As I know, only Babel has presets, and other bundlers only have a bunch of configuration fields in their documents, which needs users to set them up manually.

Any build tool can be wrapped to templated with defaults you like. For example https://github.com/antfu-collective/vitesse or https://github.com/preactjs/preset-vite

wooorm commented 3 months ago

In my mind, GPLs are for powerless individual coders, and others are for powerful companies, universities or foundations.

I don’t think this understanding is correct; npm being 99% MIT shows that “powerless individual coders” are also using MIT.

But why? No one knows the logic in it, it doesn't make sense. I will do anything for what makes me believe.

You can ask? :wink: What do you want to know?

I thought about that, but it'll be confused for users to find a plugin only support MDX 1.x in MDX 3.x document.

I don’t think I see that? I don’t think so? Why would someone be confused with a note? You can show 2 things in that section?

I’d lean towards adding a note, as it’s something for now, and Parcel will solve it later?

TechQuery commented 3 months ago

This plugin is [...] only for individuals.

This concerns me a bit.

I'm sorry, my Speak-to-Text Input Method lost the not, I mean "Not only for individuals"😂

Any build tool can be wrapped to templated with defaults you like. For example https://github.com/antfu-collective/vitesse or https://github.com/preactjs/preset-vite

If a framework or bundler needs a bunch of scaffold codes to make it good to use, it's not good enough actually.

Zero Configuration design is beyond the presets, because the Web standards are the best presets. Parcel bundles projects based on How we refer assets (build dependency graph with href, src, url(), import, etc.) and What we write (auto install compilers and official plugins with .html, .css, .less, .js, .ts, .tsx, etc.).

If users have some advanced requirements, Parcel's plugin system loads configurations of all the existed tool chains with the original ways (.babelrc, tsconfig.json, etc, as you mentioned before), doesn't like others, such as: Vite's React & Vue plugins have totally different ways to set up Babel configuration with barely no detail document, it makes users to feel exhausted in the migration between frameworks and bundlers...

The Web is simple before, and powerful but complex today. What Parcel does is to make the Web easier again.

ChristianMurphy commented 3 months ago

If a framework or bundler needs a bunch of scaffold codes to make it good to use, it's not good enough actually.

Good defaults are important, yes, so is the ability to configure. I've used Parcel, CRA, and other no config systems. There comes a point in any sufficiently large project where you need configuration. Parcel and CRA are not good at configuration/customization.

Parcel bundles projects based on How we refer assets (build dependency graph with href, src, url(), import, etc.) and What we write (auto install compilers and official plugins with .html, .css, .less, .js, .ts, .tsx, etc.).

Right, and so does every other bundler and build tool from the past 2-5 years 🙂 ESBuild, Vite, Rspack, etc all focus on aligning to web standards first and foremost, and have lots of official plugins.

The Web is simple before, and powerful but complex today. What Parcel does is to make the Web easier again.

I'm glad you're excited about web development 🙂 Keep up the passion and energy. 👍

wooorm commented 3 months ago

Zero Configuration [...]

There is a place for zero config stuff!

There is also a place for tools, the stuff we make here in unified, that make all the other tools possible. Some of those tools have different needs!

MDX, remark, unified, are configurable. And this is important. If you call your things MDX or unified, they must be configurable as well. If you want a zero-config tool, on top of MDX, you can make that, but it’s not MDX. I think you need a new name.

The Web is simple before, and powerful but complex today. What Parcel does is to make the Web easier again.

I personally don’t think the web has ever been simple. The web is about humans. Not even just humans: human--computer interaction, computer--computer interaction, human--human interaction. That’s always going to be incredibly complex.

remcohaszing commented 3 months ago

Re license: I recommend using MIT. It’s free, common, and simple. But you’re free to use whatever license you want. You can always relicense your own projects (but the old license applies to old versions). I don’t think we should block any (L)GPL licensed projects from being listed on the MDX website.

I agree with @wooorm that an MDX integration for a build tool should support MDX options. The current implementation picks an arbritrary list of plugins. Some of these are common, other’s I haven’t even seen before. I think that shows there’s the need for the user to specify their own plugin list.

wooorm commented 3 months ago

I agree with @wooorm that an MDX integration for a build tool should support MDX options. The current implementation picks an arbritrary list of plugins. Some of these are common, other’s I haven’t even seen before. I think that shows there’s the need for the user to specify their own plugin list.

Perhaps we should instead document to Parcel users how they can make their own plugin?

I think the core code is:

import {Transformer} from '@parcel/plugin'
import {compile} from '@mdx-js/mdx'

const transform = new Transformer({
  async transform({asset}) {
    const source = await asset.getCode()
    const file = await compile(source)
    asset.type = 'jsx' // Question: this is JS already, should it be `js`?
    asset.setCode(String(file))
    return [asset]
  }
})

export default transform
remcohaszing commented 3 months ago

I haven’t used Parcel before, let alone dealt with plugins. I also haven’t looked in depth too much. Don’t they support any configuration options?

Perhaps we should instead document to Parcel users how they can make their own plugin?

Or we could to it like @mdx-js/node-loader: expose a function to make it simple to create your own, and expose one with default configuration.

I think the core code is:

import {Transformer} from '@parcel/plugin'
import {compile} from '@mdx-js/mdx'

const transform = new Transformer({
  async transform({asset}) {
    const source = await asset.getCode()
    const file = await compile(source)
    asset.type = 'jsx' // Question: this is JS already, should it be `js`?
    asset.setCode(String(file))
    return [asset]
  }
})

export default transform

We should pass the file path to compile() if we have access to it. The asset type should indeed match the output type. This means jsx if the {jsx: true} option is passed to compile() or js otherwise. I would also be surprised if Parcel doesn’t have source map support.

TechQuery commented 3 months ago

@ChristianMurphy

Parcel and CRA are not good at configuration/customization.

Parcel's plugin system is different than others, but easy to configure & customize.

In the other hand, CRA's configuration/customization is suck.

Parcel bundles projects based on How we refer assets (build dependency graph with href, src, url(), import, etc.) and What we write (auto install compilers and official plugins with .html, .css, .less, .js, .ts, .tsx, etc.).

Right, and so does every other bundler and build tool from the past 2-5 years 🙂 ESBuild, Vite, Rspack, etc all focus on aligning to web standards first and foremost, and have lots of official plugins.

webpack needs <% something %> to inject JS files, others load images, videos, etc by absolute paths from public/, which can't be hinted by your IDE.

But Parcel uses relative paths hinted by IDE, and auto install all compilers & official plugins, which others do not.

TechQuery commented 3 months ago

and Parcel will solve it later?

@wooorm Parcel team has few action on MDX upgrading, "later" is undefined for years: parcel-bundler/website#1102

MDX, remark, unified, are configurable. And this is important. If you call your things MDX or unified, they must be configurable as well. If you want a zero-config tool, on top of MDX, you can make that, but it’s not MDX. I think you need a new name.

As I said above:

🚧Because MDX 3 has no official configuration file, so I'm thinking about the configuration schema of my own plugin. I'm glad to have issue or PR submitted for this feature, and hoping a simple design to keep Parcel easy.

When I resolve EasyWebApp/Parcel-transformer-MDX#1, I will rewrite the whole project with TypeScript to replace the quick & dirty version (which is made as the reason of long time waiting for Parcel MDX plugin updating).

so, a "Parcel-MDX plugin" means compile & bundle MDX with Zero Configuration first, and then it will be configurable for customization. I'm designing & implementing the configuration file, no renaming is needed.

ChristianMurphy commented 3 months ago

In the other hand, CRA's configuration/customization is [bad]. webpack needs <% something %> to inject JS files, others load images, videos, etc by absolute paths from public/, which can't be hinted by your IDE.

@TechQuery I agree, you're going after a straw-man, I'm not talking about webpack. 🙂

Parcel team has few action on MDX upgrading, "later" is undefined for years

Your code could also shared as a PR to Parcel itself to upgrade the official MDX integration, right? And if it were updated in the official repo, documentation across GitHub and the internet wouldn't need to be updated.

TechQuery commented 3 months ago

webpack needs <% something %> to inject JS files, others load images, videos, etc by absolute paths from public/, which can't be hinted by your IDE.

@TechQuery I agree, you're going after a straw-man, I'm not talking about webpack. 🙂

  1. webpack is also active developing, not too old.
  2. "others" means Vite, etc.

For now, No one latest bundlers implements all Zero Configuration features of Parcel, such as relative paths & auto installation.

Your code could also shared as a PR to Parcel itself to upgrade the official MDX integration, right? And if it were updated in the official repo, documentation across GitHub and the internet wouldn't need to be updated.

@ChristianMurphy I did, look up for the PR link.

TechQuery commented 3 months ago

The current implementation picks an arbritrary list of plugins. Some of these are common, other’s I haven’t even seen before.

@remcohaszing All preset MDX plugins come from official "Guides" document, which should be common for most users.

TechQuery commented 3 months ago

Perhaps we should instead document to Parcel users how they can make their own plugin?

I think the core code is:

import {Transformer} from '@parcel/plugin'
import {compile} from '@mdx-js/mdx'

const transform = new Transformer({
  async transform({asset}) {
    const source = await asset.getCode()
    const file = await compile(source)
    asset.type = 'jsx' // Question: this is JS already, should it be `js`?
    asset.setCode(String(file))
    return [asset]
  }
})

export default transform

@wooorm My plugin came from https://github.com/parcel-bundler/parcel/pull/7922#issuecomment-1750704973, but we shouldn't make Parcel users to write it again & again.

ChristianMurphy commented 3 months ago

which should be common for most users. we shouldn't make Parcel users to write it again & again.

Users have a really wide variety of use cases for MDX. Vanilla MDX is the most common use case.

If you want to avoid users having to write a custom code for when the do want a plugin. Support the remark and rehype configuration files so users can add the plugins they want https://github.com/mdx-js/mdx/pull/2523#issuecomment-2272252267

TechQuery commented 3 months ago

If you want to avoid users having to write a custom code for when the do want a plugin. Support the remark and rehype configuration files so users can add the plugins they want #2523 (comment)

@ChristianMurphy @wooorm @remcohaszing Custom Configuration support is released: https://github.com/EasyWebApp/Parcel-transformer-MDX/releases/tag/v0.4.0

TechQuery commented 3 months ago

Or we could to it like @mdx-js/node-loader: expose a function to make it simple to create your own, and expose one with default configuration.

@remcohaszing It would be nice if someone upgrades MDX Node.js loader to read configuration from a mdx.config.js file, then EasyWebApp/Parcel-transformer-MDX#1 may be no need to do.

wooorm commented 3 months ago

We should pass the file path to compile() if we have access to it. The asset type should indeed match the output type. This means jsx if the {jsx: true} option is passed to compile() or js otherwise. I would also be surprised if Parcel doesn’t have source map support. — @remcohaszing

all good points!

My plugin came from parcel-bundler/parcel#7922 (comment), but we shouldn't make Parcel users to write it again & again. — @TechQuery

a) ok, that linked code can be improved b) I actually do think it’s very useful for developers to learn the core technology they use, so MDX, Parcel, and how to combine them. Instead of inventing a new configuration format. Because again: everyone needs different plugins. Everyone needs that slightly differently. That also means that when Parcel, MDX, or plugins, update, users can update too, they don’t need to wait for you. Or Parcel. I think this is what Grunt and Gulp got wrong.

wooorm commented 3 months ago

It would be nice if someone upgrades MDX Node.js loader to read configuration from a mdx.config.js file, then https://github.com/EasyWebApp/Parcel-transformer-MDX/issues/1 may be no need to do.

? I do not understand this issue you created on your repo?

I am not so sure I think mdx.config.js is a good idea? There is no html.config.js either?

TechQuery commented 3 months ago

Instead of inventing a new configuration format. Because again: everyone needs different plugins. Everyone needs that slightly differently. That also means that when Parcel, MDX, or plugins, update, users can update too, they don’t need to wait for you. Or Parcel.

@wooorm I have exposed the whole MDX compile() option object directly in the early morning, no new format is invented, users don't need to wait for upstreams to upgrade.

? I do not understand this issue you created on your repo?

I am not so sure I think mdx.config.js is a good idea? There is no html.config.js either?

I want to load MDX files directly in Node.js for Server-side Rendering, but @mdx-js/node-loader has no configuration file to load custom options & plugins.

So I need a mdx.config.js just like tsconfig.json (TypeScript language does have a configuration file as a good idea).

wooorm commented 1 month ago

Thanks for your patience. I just landed this. Reworded it a bit tho.