Shared Libs Clarification

bteng22 commented 6 years ago

In the webpack-shared-libs.md front-end documentation, there is note regarding cross-project sharing:

NOTE - Cross project sharing: The biggest thing to understand for shared libraries is that this first step can be completely independent of the second entry point build step -- across:

Multiple entry points in the same project / application
Multiple entry points in different projects / applications
This means that we have a truly portable, cacheable library for an entire website or collection of sites, unlike the project-specific code splitting solution.

But how can the shared library be used between completely different projects (i.e. a website composed of multiple front-end applications/components) when the manifest is created at runtime? How is manifest shared between different projects?

I know the shared library's bundle dist/js/lib.js can be shared if we set the output target to UMD but how do we achieve this when projects are in completely different repos/builds? Am I missing something in my understanding? Any clarification would be super appreciated!!

ryan-roemer commented 6 years ago

The manifest and shared lib is created initially and could be an entirely separate project. Then independent projects take a build time dependency on the manifest.

Does that help?

bteng22 commented 6 years ago

Hmm do you mind elaborating a little more or providing a basic example? I'm relatively new to Webpack so any reference or code could help!

In my case, I'm trying to create a generic UMD library with vendor packages where I don't control the authoring. We want to initially apply this vendor bundle so that all of these dependencies will be available in the consuming environment for all our apps.

const path = require('path')
const webpack = require('webpack');
const ManifestPlugin = require('webpack-manifest-plugin')

const env = process.env.APP_ENV ? process.env.APP_ENV : 'production'

module.exports = {
  context: process.cwd(),
  entry: {
    vendor: [
      'styled-components'
    ],
  },
  output: {
    filename: '[name].[chunkhash].js',
    path: path.join(__dirname, 'dist', env),
    library: 'StyledComponents',
    libraryTarget: 'umd'
  },
  plugins: [
    new ManifestPlugin({
      fileName: 'vendor-manifest.json'
    }),
    new webpack.DllPlugin({
      name: 'StyledComponents',
      path: path.join(__dirname, 'dist', env, '[name].json')
    })
  ]
};

At the moment this vendor webpack config lives in the same repo as ONE of our consuming applications and can access the Dll JSON relatively. How would a completely different project do the same?

ryan-roemer commented 6 years ago

I’m on vacation and will only have random time here and there in December but I’ll try to find some time examine your scenario more. In the meantime, look at https://github.com/FormidableLabs/formidable-playbook/tree/master/examples/frontend/webpack-shared-libs for the separate webpack configs used in our example. The dll one is fully portable

bteng22 commented 6 years ago

Ahh sorry to bother during your vacation

Whenever you get the chance, I believe that's where I'm stuck. How is the DLL portable when the manifest is created locally. I guess I'm confused on how other projects consume the DLL manifest as a build time dependency. Can't really find any documentation detailing that step

In the example you shared the DLL manifest is created in the dist folder and the consuming webpack.config.js points to the manifest at ./dist/js/lib-manifest.json. How would projects outside this repo access this manifest?

ryan-roemer commented 6 years ago

Haha, no worries...

Step one - build manifest + shared library.

So for the example, https://github.com/FormidableLabs/formidable-playbook/blob/master/examples/frontend/webpack-shared-libs/webpack.config.lib.js produces:

and that build needs to happen first. But those inputs and output files could be a completely contained repo or package. The important things to note are that manifest is now used in other project builds and the shared lib lib.js needs to define a global that provides all the shared code at runtime to other entry points built off the manifest, which we find at: https://github.com/FormidableLabs/formidable-playbook/blob/master/examples/frontend/webpack-shared-libs/dist/js/lib.js#L1-L2

var lib_00d73d25eef8ddd2ed11 =
/******/ (function(modules) { // webpackBootstrap
// ... AND ALL THE CODE ...

Step 2+ - consume manifest and build entry point with runtime dependency on `lib.js`

Separately, there is an application config at: https://github.com/FormidableLabs/formidable-playbook/blob/master/examples/frontend/webpack-shared-libs/webpack.config.js it consumes the manifest file at: https://github.com/FormidableLabs/formidable-playbook/blob/master/examples/frontend/webpack-shared-libs/webpack.config.js#L21-L24 basically:

    new webpack.DllReferencePlugin({
      context: path.join(__dirname, "../src/es5"),
      manifest: require("./dist/js/lib-manifest.json")
    }),

That currently comes from the same parent directory, but it absolutely doesn't need to. We're going to issue a separate webpack command to build the application files (that consume the manifest) and produce:

Once we're in runtime, app1.js and app2.js no longer make reference to the manifest (lib-manifest.json) but now have a built in reference to the shared library lib.js at, e.g.: https://github.com/FormidableLabs/formidable-playbook/blob/master/examples/frontend/webpack-shared-libs/dist/js/app2.js#L86

module.exports = lib_00d73d25eef8ddd2ed11;

So for your broad application strategy, you just need one shared lib on a webpage to define lib_00d73d25eef8ddd2ed11 as a global and all the depending entry points will use that as a runtime dependency.

Hopefully that better illustrates the "separateness" of the steps (?)

bteng22 commented 6 years ago

Wow @ryan-roemer thanks so much for the clarification! That all makes sense.

I think the last piece I need is how external entry points will use that lib as a runtime dependency? I tried Webpack externals but for some reason the dependencies are undefined during runtime even when they're globally visible. Do I need to use something else like ProvidePlugin?

Thanks again for all the help—you're really saving me here!

ryan-roemer commented 6 years ago

Sure.

The key part is this:

    new webpack.DllReferencePlugin({
      context: THE_USUAL,
      manifest: require("PATH/TO/YOUR/separately-built-manifest.json")
    }),

in a config for an entry point. That is what (1) leaves mappings instead of huge chunks of code for things in shared library, and (2) instructs webpack to write in the critical module.exports = [SHARED_LIB_NAME]_[HASH]; part.

You may just want to pull the code samples in this project for shared libs and hack them up a bit to see how they work together as a super simple example, then go back to your project. Good lukc!

Side note -- I don't use the webpack-manifest-plugin, so don't know how it works or if it's compatible with actual built-in webpack DLL stuff. The only two plugins we use are:

webpack.DllPlugin to produce the manifest + shared lib in a preliminary build step / possibly separate project
webpack.DllReferencePlugin to consume the manifest and instruct webpack to write references to the correct global variable corresponding to the shared library export in a later entry point build step.

bteng22 commented 6 years ago

@ryan-roemer you're incredible! Thanks so much for all the help 😄

Just to make sure I got this right, if I wanted to share the manifest + shared lib produced by the webpack.DllPlugin with separate projects, it would have to be a preliminary build step in order to share the manifest that would be consumed with the webpack.DllReferencePlugin, since it's a build time dependency.

At the moment, I got everything working on my end, but by copying the contents of the manifest.json and sticking it into my secondary application. The primary application is the one that is exposing the module.exports = [SHARED_LIB_NAME]_[HASH]; as a global for the secondary application to use as a runtime dependency. Now I just need to extract my lib webpack with the DllPlugin.

Thanks again for clearing up all the ambiguity on my end, and hope you enjoy the rest of your vacation. Cheers 🎆

ryan-roemer commented 6 years ago

Cool! Closing this issue, but just ping us if you need more help. Good luck!

FormidableLabs / formidable-playbook