[RFC] New Asset support via Plugins or Core?

[davidnagli]

💬 RFC

Do you guys think support for new things should be added to the core, or via plugins?

Currently, we have a weird mix of both:

• Svetle support is via a plugin (thanks to @DeMoorJasper)
• Stylus is in the core
• TypeScript is in the core
• etc

What do you guys think?

To sum up what I heard in the slack:

• Perspective 1 - Keep the core lean, we should have a lean, minimal, and performant core. We should rely on plugins to integrate with all “non-standard” file extensions (so things like .jsx vs .js)
• Perspective 2 - If we have everything through plugins, we aren’t living up to our mission of being 0 config. We end up being the same as Webpack, where users need to install a billion plugins
• @albinotonnina Thinks that we should use plugins, but provide an easy CLI tool for importing them

It’s probably going to end up being a combination of all of these…

If so, we need to decide which files should be considered “core” and which “plugins”

[davidnagli]

What are your guys’ thoughts?

[dschnare]

And if asset types are indeed going to be part of core, a way to list them. I tried looking for TypeScript support and I didn't know it was in core until reading this issue.

[albinotonnina]

To elaborate more on that, I imagine something along the lines of:

• on Project Init, somewhere after my init command:

Image from Inquirer

• Later on $ parcel supports jpg svg

and maybe utilities like: $ parcel supports --list etc

I would like this sort of granular control on this.

[EduardoRFS]

"The core", don't need to be the "parcel-bundler" package, perhaps the default setup contains a "lot" of plugins, like stylus, ts, jsx, and exists a "clean" package without everything, perhaps even the default css, html can be removed.

But for the tool supports a default option to setup vue and react fully support it's a cool feature

[artyomtrityak]

what about Flowtype? Pracel uses Babel preset-env, can babel-preset-flow be added too?

[davidnagli]

@artyomtrityak This doesn’t really have to do with Parcel plugins… Parcel supports .babelrc, so you can add any plugins you want.

So what you said isn’t really relevant to this RFC 😇 😜

[jamiebuilds]

I think that assets should probably be implemented exactly the same as plugins even if they are built-in. That way they people can use them as references

[brandon93s]

I think that assets should probably be implemented exactly the same as plugins even if they are built-in. That way they people can use them as references

See #251 for an example of this. Having core assets (and base types) implemented as plugins would enable these. They could just be included installed by default in parcel.

[davidnagli]

@thejameskyle That’s a pretty good idea. But generally how should we implement support for new Assets? Should we make these plugins part of the core, or separate?

[davidnagli]

I think I have a solution! It would allow us to have both a lean and minimal core, while allowing extensibility to a variety of asset types, while maintaining our zero config philosophy 😃

Here’s how my idea works:

• Only keep standard assets in the core
• Keep all non-standard assets in a separate, but officially maintained repository
• When the user runs Parcel, we automatically detect which asset-types that they don’t have installed, and install them.
  • For performance, we should cache these installed assets. So, if the user has Parcel installed globally, these cached asset plugins can be shared globally to make the process instant on subsequent projects.

At first thought, this idea of downloading plugins dynamically during build-time might seem to be prohibitively expensive. But we really only need to install these dependencies once (we can lazy-update them in the background whenever that’s necessary). Also, we can have our CLI clearly show the user that we’re installing it for them, so they won’t think that this is Parcel being slow.

Additionally, 99% of projects won’t have 200 different non-standard assets. In reality, the majority of projects will not have more than 2-3 non-standard asset types at most, making the cost of dynamically installing non-standard assets in this way less of an issue, especially considering it’s a cost we’ll likely only need to pay once.

If you think about it, anyway the user is going to need to install the asset one way or the other, it’s just a question of whether we force them to manually install it once they need it (make everything plugins), automatically install EVERYTHING right away when they first install parcel (make everything core), or if we dynamically install plugins when they need them, without any config/setup.

  Note: I made these terms up, but:

• standard assets are assets for files that are already officially supported by the browser:
  • I think standard assets should be part of the core
    1. JS
    2. CSS
    3. HTML
    4. etc…
• non-standard assets are assets for files that aren’t officially recognized by the browser or part of the web spec (3rd party stuff)
  • I think non-standard assets should be plugins
    1. JSX
    2. TypeScript
    3. SASS
    4. Stylus
    5. etc…

[EduardoRFS]

hmm it's work, but ... isn't better at least the most popular in core? Like JSX? Isn't like 10-20 default assets gonna be so expensive in core

[DeMoorJasper]

Sounds like a solid idea We could probably use yarn for installing these plugins because then we don't have to worry about caching/installing or anything else. Only thing i'm mainly worried about is performance, everytime u run parcel it would have to check if u added any new assets with a new extension that has not been installed yet.

I don't know in how far this is possible but maybe we could install them while processing assets, like process the assets we can already process meanwhile install any plugin that needs to get added for the non-standard assets? Once a user adds a new asset that's not already installed, we could again install it and then compile even while running in hmr?

[davidnagli]

Only thing I’m mainly worried about is performance, everytime u run parcel it would have to check if u added any new assets with a new extension that has not been installed yet.

@DeMoorJasper No we don’t! 😃 We just run parcel as is, and if it fails due to an unknown asset type, then we actually go ahead and try to install the corresponding asset plugin. This way we can avoid the overhead of trying to detect new asset types.

[DeMoorJasper]

@davidnagli will still cause worse performance but would actually only impact user if they added new extensions and once cache kicks in + all plugins installed it would actually work pretty insane fast. I'll have a spin at this way of working by first testing it on localrequire, we might even register all known extensions and localrequire the plugins as well 🤔 (than we are zero config/zero install?)

Although i just realised how do you detect all languages that use the same extension?

[brandon93s]

I'd like to propose an alternative, that may be a bit more "long-term":

All Asset Types are Plugins

1. Asset.js and all asset implementations are extracted into their own repositories in the parcel-bundler organization. This includes their testing and necessary dependencies. This will give you repos such as:

• parcel-bundler/asset
• parcel-bundler/html-asset
• parcel-bundler/css-asset
• etc

2. All asset types are published to NPM under the @parcel-bunder organization namespace, individually:

• @parcel-bundler/html
• @parcel-bundler/stylus
• etc

3. All "core asset types" (officially supported by the parcel team) are yarn add (installed) into core (this repo) by default, as dependencies. Parcel core could reference latest (npm release) by default so it always pulls the newest version.

4. Parcel is enhanced to look at @parcel-bundler/* dependencies (in addition to parcel-plugin-*) and treat them as plugins / register. Plugin resolution would be treated in the following order:

• add core-installed @parcel-bundler plugings
• add all plugins in user's project (parcel-plugin-* AND @parcel-bundler/*)

Advantages

• Allows a user to import @parcel-bundler/asset base for plugin development (#251)
• Allows for assets to be worked on, maintained, and released independently
• Contributions become less intimidating, given concise and targeted chunks of the project
• Allows a user to "pin" a version of an asset to a specific version independent of core and other assets, since they could yarn add @parcel-bundler/html@1.0.1 which overrides parcel's internal implementation.
• Remains "zero-config" since core assets are installed in the base module

Community Plugins

User land plugins can extend directly off of existing assets without the need to require the entire core bundler.

Plugins with enough community support / demand, can be installed into core as dependencies. This allows the entire community to benefit. Optionally, they can "graduate" into the organization as official assets at the creators discretion.

Instead of auto-installing or any thing along those lines, provide a sensible way to search for asset types that aren't supported out of the box (e.g. user-land plugins):

$ parcel search buckle
The following plugins match your search, use `yarn add {name}` to use them with parcel:

   1) parcel-plugin-bucklescript - A parcel plugin that enables bucklescript support

This would just search the npm registry.

[rondonjon]

I think a "core" should be as small as possible (that's why it's called a core), and all assets, plugins, etc. should be treated equally and separately.

Every attempt to define an "essential" set of those is inherently biased and somewhat arbitraty. A Typescript programmer might find js useless, a routined scss user will have little need for plain css. Hence it should be considered as "normal" and not as an "edge case" that people pick their extensions, and possibly also need to do that manually. It's quite a common thing, right? For babel, react, and many other popular software packages, user-selected npm i is actually the way of mapping individual needs to available features.

Yet, if a majority thinks that pre-bundling is indeed necessary, it could be an option to have something like distributions of parcel, which could then include different feature sets, e.g. just the core plus essential tools ("minimal"), or the prior plus a curated set of extensions ("standard"), or the prior plus all other officially maintained extensions ("full").

About the automatic downloads, I am not sure. It seems very convenient and seamless at first sight, but I am unsure if users actually want that, and where it could fail for reasons beyond a simple standard setup. For example, many companies use internal repositories that often require authentication. If yarn was used to silently download missing dependencies, people might need to preconfigure it appropriately, and in such cases, manual postinstallation of additional dependencies once within every couple of months really seems more straightforward. Just imagine having to write test cases for the many weird setups that people have.