[Proposal] Update `webpack` to v5

[joshuarrrr]

Much of the js bundling tasks are currently handled by the osd-optimizer package, which uses webpack and babel (via babel-loader) to build plugin-specific bundles. This is also the process by which plugin-specific dependencies and project-wide dependencies are resolved and bundled together.

osd-optimizer has a dependency on "webpack": "^4.41.5" (as do other OpenSearch Dashboards packages that depend directly on webpack, because packages are not allowed to have diverging versions of the same dependency). That version is nearly 3 years old, with the most recent webpack version being 5.6.0. If we plan to continue to rely on webpack for bundling it's likely worthwhile to upgrade to version 5, because we've already run into instances where version 4 struggles to handle common ES6 syntax in javascript files.

But it's also a bit unclear what our future bundling strategy will be. The README of osd-pm implies that the previous developers planned to eventually drop webpack altogether:

https://github.com/opensearch-project/OpenSearch-Dashboards/blob/b19c1717a29fffa1b40f8073c6dfb3fb4fe85972/packages/osd-pm/README.md?plain=1#L10-L13

This question may also have implications for plugin decoupling and the extensions initiative. I'm also not sure to what extent our solution here will be related to questions over the future of the osd-pm package.

[joshuarrrr]

@AMoo-Miki @ashwin-pc @kavilla I'd appreciate any feedback or additional context you can provide on this topic.

[mihirsoni]

Ideally IMO, we should step back and look for long term, this may become major upgrade for OSD. I would evaluate the other option and see if that makes sense for us and can be more community friendly. lIke Rollup.

[BSFishy]

The README of osd-pm implies that the previous developers planned to eventually drop webpack altogether:

IMO, we shouldn't deviate from standard tools. By using tools that the community is already familiar with, like Webpack, the project is a lot less intimidating to newcomers because there are already parts that they know and understand.

I think a short-er term goal could be to upgrade to Webpack 5, as we already have a lot of configuration and things for Webpack, and like @mihirsoni mentioned, we could take this as an opportunity to see if Webpack still fits our long-term goals for OSD. There are a lot of bundlers and compilers out there, and some might fit this project better than Webpack.

[joshuarrrr]

@mihirsoni Can you explain a bit more about why you think this would be a major upgrade? It seems like bundling today is largely an implementation detail that doesn't directly affect all developers in a direct way.

and can be more community friendly. lIke Rollup

Can you clarify what are the important criteria that make a given bundling approach more or less community friendly? Is it the approachability/familiarity of the actual bundling tools themselves or the way they're integrated into the projcet (osd-optimizer, osd-pm), or both?

[kavilla]

I think one of the reasons historically is that we have this monolith repo and it has to be optimized to build and share dependencies. But then it also has externalized code. We can also look if we are actually changing some aspects of how we source control these items. If break apart this repo do we also need any optimization for a monolith repo?

It osd-pm also inspired by Lerna working with yarn workspaces so should we also figure out how much we have deviated from Lerna?

[joshuarrrr]

Yeah, @kavilla I think our research and decisions around the future of osd-pm can continue in https://github.com/opensearch-project/OpenSearch-Dashboards/issues/294. Here, I'm mostly trying to figure out if there are any particularly strong reasons for us to not go ahead and upgrade webpack within our current bundling framework. (Or if we need to wait for more research and knowledge.)

[ashwin-pc]

A webpack upgrade was attempted in the past for 2.0 #1118. The refactor involved was apparently significant. Identifying if thats still the case will determine the path forward here. If it is a big lift, we should evaluate our plan long term instead of upgrading since it will be a lot of throwaway work. If we can relatively easily upgrade it, a short term plan that involves an upgrade is possible.

[joshuarrrr]

😆 @ashwin-pc Thanks for linking #1118 - I don't know why I didn't search for it! For what it's worth, I think I'm fairly close to a working PoC for the upgrade that won't be too bad, but I'll definitely timebox it to estimate feasibility.

[tmarkley]

@joshuarrrr did you finish a working PoC? I'm curious to hear about how it went!

[joshuarrrr]

No, not yet - I got stuck enough I needed to set it aside for a bit to just unblock the vega-lite upgrade.

[neodescis]

I've been trying to incorporate an Angular 15 standalone component into a custom plugin visualization. Theoretically this should be possible. It looks like someone has created a simple React wrapper around Angular 14.2+ standalone components.

I tried to achieve this in an OpenDashboards plugin, but ultimately webpack refused to resolve many of the @angular ES modules. I was able to get some of them working by adding .mjs support (including type: 'javascript/auto') to the osd-optimizer webpack config, but then I was seeing imports from @angular modules to other @angular modules couldn't be resolved. I believe this might be a webpack 4 issue? At any rate, I attempted to update things to webpack 5. I got a fair way down the rabbit hole, but I've had to throw in the towel when I got to BundleRefsPlugin and I was seeing lots of TypeError: Invalid value used as weak map key messages.

[plin13]

hi, I'm curious if there's been any progress or further discussion on this webpack issue.

[ashwin-pc]

There were multiple to do this migration a few times in the past but each time we ran into issues given the custom build system for OSD. @plin13 is this blocking you in some way right now?

[plin13]

In a way yeah. We're using opensearch and dashboard on an enterprise level and webpack 4 is quarantined due to security risk.