search

aurelia / webpack-plugin

A plugin for webpack that enables bundling Aurelia applications.
MIT License
90 stars 36 forks source link

Webpack 5 support #166

Closed mlassbo closed 3 years ago

mlassbo commented 4 years ago

I'm submitting a feature request Add support for Webpack 5.

We (at Hogia Group) need to start preparing our Aurelia apps for Webpack 5. It is in beta but Webpack reccommends to use it for production. Also, Webpack 5 comes with Module Federation that might become an important part of building micro frontends using Aurelia and we need aurelia-webpack-plugin to support webpack 5 to be able to evaluate this properly. I've been in contact with webpack core member Zack Jackson @ScriptedAlchemy, the creator of Module Federation, about this and he has offered his help with the upgrade if needed.

Please tell us about your environment:

Current behavior: When upgrading to webpack 5 beta 17 build fails with the following error:

Error: Cannot find module 'webpack/lib/BasicEvaluatedExpression' Require stack:

Expected/desired behavior: Build runs without errors

jods4 commented 4 years ago

I wrote the Webpack 3 and 4 versions of the plugin but unfortunately I don't have time to upgrade it to 5 at the moment.

I might do it later on when I'm less stressed, although if someone wants to submit a PR that fixes APIs that have changed in Webpack 4 you're welcome.

A few things that come to my mind right now:

  1. We need a strategy regarding compatibility with Webpack 4. My thinking is: Aurelia 2 won't need this plugin as it is designed from scratch to be modern-tooling-friendly. This plugin is unlikely to receive big changes at this point. So I'd just give up compatibility with a major release. Webpack 4 users can continue to use aurelia-webpack-plugin@4.0.0 without loosing anything and Webpack 5 users take a dependency on aurelia-webpack-plugin@5.0.0

  2. The module federation thing might be a different beast, just like concatenated modules required extra changes. Or maybe not, but it's really not a given. aurelia-loader has some hard requirements when it comes to module names and this plugin is all about making sure every module ends up with the right name. Guaranteeing this in separate builds might require some tweaks. I agree it's a very interesting feature that I would like to see supported as well!

fkleuver commented 4 years ago

My thinking is: Aurelia 2 won't need this plugin as it is designed from scratch to be modern-tooling-friendly.

That's mostly correct. Conventions do rely on a thin layer of tooling, but without conventions you don't have to do anything and Aurelia 2 will indeed work OOTB with any bundler including webpack 5.

However, since conventions in au2 is just a simple pre-processing step that adds decorators, I don't imagine much (if any) difficulty at all in supporting that.

On the note of au2, we are getting close to alpha. Is upgrading to au2 something you'd be interested in on the short to mid term as an alternative, or are there strong reasons to stick to au1 for some time to come?

jods4 commented 4 years ago

However, since conventions in au2 is just a simple pre-processing step that adds decorators, I don't imagine much (if any) difficulty at all in supporting that.

Yeah this could be a simple webpack loader.

Au1 needed complex stuff to inform webpack of the dependency graph + ensuring everything would be properly available to aurelia-loader, which is purely based on module names.

Is upgrading to au2 something you'd be interested in on the short to mid term as an alternative, or are there strong reasons to stick to au1 for some time to come?

I'm assuming this question was for OP.

mlassbo commented 4 years ago

Thanks for your responses!

For our Micro Frontends we could (and should) definitively go for au2 right away if its production ready. When it comes to our monolith au1 apps I cant really say. Some of our teams might upgrade right away and others move more slowly. Perhaps we can find a way to make Module Federation work nicely between Micro Frontends even if they are hosted in an au1 app bundled with webpack 4.

Got any examples of an au2 app bundled with webpack?

ScriptedAlchemy commented 4 years ago

As one of the core maintainers of webpack and the creator of module federation. I'm willing to work with project authors if you need me

fkleuver commented 4 years ago

Happy to hear that @ScriptedAlchemy, and great job on the Module Federation. We're very excited about it!

The compatibility issue in a nutshell: Aurelia version 1 is built on top of a requirejs-based loader architecture that stems from before ES Modules were standardized. We use a special marker function PLATFORM.moduleName('./path-to-module') that our webpack plugin finds in the AST and turn them into a shape that webpack understands (I didn't write the plugin but I believe it transforms them to native imports, @jods4 correct me if I'm wrong).

Unless you're aware of any specific concerns off the top of your head, I think we're mostly looking at simply try to upgrade to webpack 5 and see what breaking changes we need to address and take it from there. If @jods4 is too busy I can have a crack at it later this week.

Got any examples of an au2 app bundled with webpack?

Scattered around at the moment. I would recommend to wait for alpha announcement (within a few weeks), which will come with significantly more docs and examples than we currently have.

jods4 commented 4 years ago

that our webpack plugin finds in the AST and turn them into a shape that webpack understands (I didn't write the plugin but I believe it transforms them to native imports, @jods4 correct me if I'm wrong).

It doesn't work at the syntax level. It turns them into native, custom webpack dependencies using the "internal" webpack apis. Those dependencies have several special characteristics such as not being eligible for merging with another module (except if they are root of the resulting module), always preserving their name, etc.

moduleName is not the only type of Aurelia dependency with need to detect. Examples would include the implicit association between Resource.js and Resource.html, <require> used in an html template, etc.

fkleuver commented 4 years ago

It turns them into native, custom webpack dependencies using the "internal" webpack apis (...)

I see, that sounds like it might be a bit more work then.

Examples would include the implicit association between Resource.js and Resource.html, used in an html template, etc.

But those are AST transforms right? Or at least that's what they are in the au2 conventions plugin, so if they currently aren't, maybe we can simplify things a bit by doing so, if that makes sense?

jods4 commented 4 years ago

But those are AST transforms right? Or at least that's what they are in the au2 conventions plugin, so if they currently aren't, maybe we can simplify things a bit by doing so, if that makes sense?

Not really. Because the "things" are not loaded directly in code with an import, rather they are loaded dynamically at runtime by aurelia-loader, which works in a peculiar way with Webpack.

This was one of the main driver for design changes in Au2, I'm afraid we did the best we could to retrofit Au1 design into the webpack era.

fkleuver commented 4 years ago

I'm afraid we did the best we could to retrofit Au1 design into the webpack era.

What about dynamic import era? AFAIK this plugin was implemented way before that was available, but now it so happens to be that import('./some-resources') precisely implements the LoaderPlugin interface

interface LoaderPlugin {
  fetch(address: string): Promise<any>;
}

Dynamic imports is something webpack understands OOTB. Has that by any chance been looked into?

jods4 commented 4 years ago

What about dynamic import era?

If you do everything with dynamic imports you probably won't need this plugin and can go with vanilla Webpack (it was the goal at least).

We can't expect all our users to migrate their codebase to that new syntax. I know I won't. So supporting "old-school" Aurelia is pretty much a requirement.

fkleuver commented 4 years ago

We can't expect all our users to migrate their codebase to that new syntax

I think I wasn't clear in what I meant, sorry. I mean that we use the new syntax on our side in some fashion. For example, by doing an AST transform from PLATFORM.moduleName() to import(). If I'm not mistaken, webpack is smart enough to figure out things that the browser can't, like adding the file extension and the /index convention. Does that make sense?

jods4 commented 4 years ago

I see. This would be crazy hard to do, impossible in some cases.

Let's say I have this module in my project:

export const route = { name: 'home', module: moduleName('./home.ts') };

export class HomeViewModel { }

You'd need to figure out in what Aurelia API the route.module export is gonna end up if you want a chance to make a meaningful transformation. It can mean changing the API on the other end of the pipe. In general cases it's impossible.

webpack-plugin has cases that are not even linked to actual source code. In my project I use GlobDependenciesPlugin to ensure everything in my ViewModule folders is considered an Aurelia module and included -- I do not use moduleName in my code.

fkleuver commented 4 years ago

You're right. The transform suggestion must have been a brainfart moment on my part, for some reason I thought of it as a string-to-promise function but of course it's just string-to-string and has nothing to do with imports. My bad.

My original thought was an implementation of aurelia-loader where the load method simply calls import(), but that won't help us either since webpack can't statically analyze dynamic expressions.

So to come back to the complexity problem of doing the transformation that you thought I meant just now - we might be able to do what with v2's AOT once it's done, because that's built as a more or less fully fledged ESNext runtime that evaluates everything that's reachable (it bails only at things like fetch calls and DOM input bindings). That's at least several months away though.

In the mean time, the quickest way for us to achieve this webpack interop is probably to add import-friendly API's in places where it's still missing. @bigopon already added some static API's for templating and routing so we might already be there. That does mean users will have to migrate their apps to using dynamic import syntax if they want to use module federation - I wonder if that's a type of investment they're willing to make as an addition. @mlassbo ?

If that's acceptable then we don't need to touch the v1 webpack plugin beyond ensuring general v5 api compatibility. Of course we can try to look at module federation specifically, but I'm worried we might be opening a can of worms of something that seems fine in the beginning but issues keep popping up. Going native-compatible really feels like the safer bet.

Eventually we can pull v1 along with the more advanced AOT tooling of v2, but again, that's at least several months away. It's all about what we can offer in the mean time and make the right trade-offs.

mlassbo commented 4 years ago

For our sake there's no rush to make this work with v1. If we can be rest assured there will be support for webpack 5 and module federation for v2 we can continue evaluating the technology to see if it's a good fit for our micro frontend challanges.

jods4 commented 4 years ago

I'm a little busy right now but I want to bring the plugin to Webpack 5. I'd like to support module federation and I think it should be possible (maybe even easy) but there's no guarantee until we try and find out.

@fkleuver A key goal of v2 should be to get rid of aurelia-loader and instead rely on import or stuff that can easily be transformed into import by a basic loader. This means full compatibility with every JS bundler out there without plugins or crazy hacks.

fkleuver commented 4 years ago

@jods4 Yes, v2 has been aurelia-loader-free from the get-go. We only have a thin (optional) transform layer to apply conventions, and all that does is add the import statement and @customElement decorator to the component if there is a same-named html file and the decorator isn't there already. It also transforms <import> tags to import statements.

The AOT I'm talking about is primarily for build-time optimizations and a few minor quality-of-life improvements, such as making autoinject work with interfaces as well instead of just classes. No crazy hacks there in order to make v2 work in the first place, just optional stuff to further improve things.

ScriptedAlchemy commented 4 years ago

Implementation of MF has been pretty easy with other OSS upgrades so far. So I’d imagine leveraging it will be fine as long as you can run WP5 :)

luboid commented 4 years ago

Hi there, It will be nice if plugin has support for externals in WebPack 5:

https://webpack.js.org/configuration/externals/

@jods4

jods4 commented 4 years ago

@luboid I don't think externals require support from this plugin.

luboid commented 4 years ago

@jods4

It will be something needed in the Micro Frontend world. When you have a shell (Aurelia app) that loads different modules (Micro Frontends) as Web Components (Aurelia), here can be used also IFrame to load the standalone app (for isolation and unload code). This is one of the possible ways to achieve the Micro Frontend pattern. And you want to share resources between them with browser cache. Shareable resources (like Aurelia runtime, etc) can be placed JS Libs, by this you can easily fix bugs in vendor's things and vise versa. The application can be left only with the module (App) domain logic code.

We now have around 26 modules on DurandalJS, every one of them comprises one bundle for Durandal itself, one bundle for the module (App) and other major libs are loaded on demand by RequireJS, when a module needs them. And that is wrapped around with Shell.

bigopon commented 4 years ago

@luboid I think what you want can be achieved by a combination of Webpack exposing Aurelia core modules & provide plugin to alias to those globals in the micro apps

luboid commented 4 years ago

@bigopon yes, I think that is the point of WebPack Externals to alias global libs (umd, etc), more interesting is that they have some experimental feature to load them async, however. "Webpack exposing Aurelia" I think this needs to be a most likely builder which to pack Aurelia core modules or to inspect some projects and to build intersect between them, I don't know.

jods4 commented 4 years ago

@luboid Bundling "other major libs" as externals should work without problems. Bundling anything that contains a resource loaded by aurelia-loader (which basically includes all Aurelia resources such as custom elements, custom attributes, value converters, etc.) is not gonna work without some runtime support from aurelia-loader.

Some alternative APIs that are more bundler-friendly were introduced at some point, which may help getting this to work if you tweak your app to use the new APIs. I am not sure if this was properly documented, @bigopon do you have any pointers?

Given the current focus on Aurelia 2, which is built from the scratch to be bundler-friendly, I think it's unlikely that the runtime capabilities of Aurelia 1 change significantly at this point.

luboid commented 4 years ago

@jods4 Then I will be waiting to see what will come with Aurelia 2

I'm not sure that I'm clear, because of that I will post one link, go and see Step 6: Sharing Libraries Between Micro Frontends

deleonio commented 3 years ago

When should Aurelia v2 come?

deleonio commented 3 years ago

PR: https://github.com/aurelia/webpack-plugin/pull/169

deleonio commented 3 years ago

the same story here:

Many thanks to Angular for influencing Webpack and the unconsolidated upgrade to v5.

graycrow commented 3 years ago

Can someone take a look at this? Every day more and more packages are dropping support for Webpack 4. Today I have obsolete webpack, webpack-cli, webpack-bundle-analyzer, sass-loader, less-loader, postcss-loader, expose-loader, html-webpack-plugin, copy-webpack-plugin and this list is growing every day. Aurelia 2 may be released soon, but many large projects will not be able to update immediately and, as a result, will be locked with outdated packages for months or even years.

Please! 🙏

ckotzbauer commented 3 years ago

@graycrow There is a open PR for this #169. It's not ready yet, but input is always welcome. 😉

ItWorksOnMyMachine commented 3 years ago

What about dynamic import era?

If you do everything with dynamic imports you probably won't need this plugin and can go with vanilla Webpack (it was the goal at least).

We can't expect all our users to migrate their codebase to that new syntax. I know I won't. So supporting "old-school" Aurelia is pretty much a requirement.

@fkleuver, if we were willing to migrate to dynamic imports with au1, could we? How would we replace PLATFORM.moduleName when supplying moduleId for routes?

jboysen commented 3 years ago

What is needed to get this working?

Because of this there must be a lot of Aurelia applications that have very old dependencies by now, and I don't think just relying on everybody upgrading to Aurelia 2 is a solution. Or whatever the reason is for why this hasn't been fixed yet.

expaso commented 3 years ago

Same here, wanting to upgrade, but can't! Please make sure we first can upgrade dependencies before we upgrade to au2. This is crucial!

bigopon commented 3 years ago

I'll start looking into this and try to resolve this

thomas-darling commented 3 years ago

@bigopon Thanks, very much appreciated! We've got some complex apps here, that we just cannot update to Aurelia 2 in the foreseeable future. So it's super important for us that the v1 toolchain remains up to date 🙂

jboysen commented 3 years ago

We’ve in the same situation as @thomas-darling, so we really appreciate this @bigopon. We are also ready to assist with any help you may need, just reach out.

Devvox93 commented 3 years ago

@bigopon Not to rush you, but do you have enough info to make a rough estimation of when this could be ready? Also, please let me know if there's a way I can help (by testing maybe?) :)

bigopon commented 3 years ago

Cant really give any update yet atm. Im atively looking into it. Maybe in a few days ill give an update of what the changes could be. Also will publish something that can be tested as soon as I can

Update 1: I'm reading the webpack v5 source code & debugging the webpack v4 with our plugin to see what changes are needed. Have been able to understand more but still not enough to refactor the plugin. I'm positive still 🤞 Update 2: After some more investigation, it seems the majority of the current plugin is still applicable. There'll still be some internal poking & hacking, but it's not too painful

jboysen commented 3 years ago

@bigopon, again, we really appreciate this. Could you give any insights into your findings, and if we can help in any way? Cheers

bigopon commented 3 years ago

I'm swapping pieces of our impl with the equivalence in v5. Will come to module.reasons soon, and push it up somewhere. Will update more.

bigopon commented 3 years ago

Here is my current draft for the needed changes: #170. I've listed work needed for all the plugins in 3 simple parts for now: typings & code & comments. Overtime, more details will be filled in. Will update my progress there as I continue the refactoring.

bigopon commented 3 years ago

An update:

With https://github.com/aurelia/webpack-plugin/pull/170/commits/fdae3aab2238bbf61a6269853540a820eb10f115, it now can build correctly with normal usages. Test app configuration can be seen at

What left to be tested:

cc @jods4

bigopon commented 3 years ago

The work can be tested by installing the plugin from the branch in that PR, the dist files have been built, can be used directly.

mlassbo commented 3 years ago

Nice work! I ran a quick test with fdae3aa and webpack 5 on one of our smaller apps (routing but no chunks) and got everything working fine. Will give it a go with a more complex app soon.

bigopon commented 3 years ago

@mlassbo thanks. With the latest commit. It should be able to cover all range of usages now, at least I hope so. Will merge and do a beta release soon.

elitastic commented 3 years ago

Thanks for your great work, @bigopon!

Looks like the aurelia-bootstrapper package is not yet compatible with webpack 5:

ERROR in ./node_modules/.pnpm/aurelia-bootstrapper@2.3.3/node_modules/aurelia-bootstrapper/dist/native-modules/aurelia-bootstrapper.js
Module not found: Error: Can't resolve 'aurelia-pal-nodejs' in 'C:\my-app\node_modules\.pnpm\aurelia-bootstrapper@2.3.3\node_modules\aurelia-bootstrapper\dist\native-modules'
 @ ./src/main.ts 15:29-60

If I analyzed correctly, "process.browser" is no longer available in webpack 5, that's why aurelia-pal-nodejs is tried to be loaded instead of aurelia-pal-browser. :(

bigopon commented 3 years ago

@elitastic did you get this issue on a plain new project, or an existing one? If it's an existing one, then it sounds like some thing else, rather than an issue in the plugin entirely. If everything is as expected, it probably shouldn't reach to the process.browser check.

MaximBalaganskiy commented 3 years ago

I’ve had the same error when tried to remove aurelia plugin completely. Webpack recognises “.require()” string suddenly and tries to bundle nodejs package

elitastic commented 3 years ago

It's an existing project. Two further, perhaps relevant disclosures:

new AureliaPlugin({ aureliaApp: undefined }),
import { bootstrap } from 'aurelia-bootstrapper';

bootstrap(async (aurelia: Aurelia) =>
{
    aurelia.use
        .standardConfiguration()
        ....
}

Could this be a problem?

MaximBalaganskiy commented 3 years ago

@elitastic the reason is webpack trying to bundle nodejs package. Olds aurelia plug-in does something to suppress that