[bundle_dir] not being replaced by python-webpack

[txbm]

Not sure if this is a python-webpack issue or not but I'm filing it here since I was able to patch it locally by replacing the [bundle_dir] string in the generate_config_for_component function with a manual join of the BUNDLE_ROOT, OUTPUT_DIR and BUNDLE_DIR settings by accessing python_webpack.conf.settings directly.

Couldn't really find the area in webpack where the replacement happens and don't have time right now to dig into it deeper but it was causing python-react to write out a config file that still had the [bundle_dir] string hard coded into the config and therefore the renderer could not find the actual component because it was getting written to a directory on the root called "[bundle_dir]".

[markfinger]

Hmm, strange. It's probably somewhere in the wrapper. Might be a versioning issue or somesuch.

The current dev version of python-webpack has done away with the bundle_dir token, it just overrides the output.path prop before the compiler touches anything.

In any case, I'll leave this open as it may be of use to someone else. I'll close it when another release gets pushed.

Thanks, for the heads up, @petermelias

[txbm]

@markfinger Yeah no problem. Axing the token is what I would have done anyway.

Have you played with the idea of allowing the user to create a webpack.config.js template that just gets dynamically populated with the per-component values? I couldn't see a straight-forward way to inject my own config without hacking around in the connecting code between python-react and python-webpack.

[markfinger]

Yeah, sort of.

A very old version used to have a class-based system which mapped attributes to webpack config properties. It proved to be pretty limiting as most use-cases end up needing a lot of JS stuff, which is difficult to handle in a python system.

After that, I implemented it as a bunch of templates. It was a nightmare to test large strings which have a dozen permutations.

So, the current system mixes JSON-serializable objects with strings. Most of a webpack config's structure is dicts and lists, and it's only when you need imports, regexs, etc that you need to throw in a string.

I've intentionally left all the config file stuff undocumented. It was a bunch of quick hacks, and I wanted a few more use cases before I settled on something.

Hmm, I've been musing over replacing a lot of our old-school asset pipelines with webpack, so I'll probably end up using the current system a bit more and - if it goes ok - I'll document things.

[markfinger]

https://github.com/markfinger/python-webpack/blob/master/tests/test_config_files.py#L50-L60

Syntactically, the current system's ugly as all hell. But functionally, I've yet to find a better solution.

Testing strings is not fun. Testing objects is easy

[txbm]

Yeah I would not want to test rendered configs using string comparison / matching. That is a complex and brittle test structure. Luckily, you might just side-step the issue of testing configs entirely.

It occured to me earlier that having python-react generate a config at all may be out of scope. Rationale as follows:

The reason I got to thinking about this is because every Webpack config I've ever seen is highly customizable. Some times you even end up writing JavaScript functions that generate your webpack config based on a whole host of environment specific variables and concerns.

It seems to me like the only way to preserve the flexibility inherent in Webpack's purpose is to basically piggy-back onto webpack.config.js generation by allowing for value injection (from Python) but not trying to build the entire config within python-react.

I picture something as simple as defining {{ py-react:KEY_NAME }} replacements inside of a user-supplied template config file that is just run through a templating engine and then executed before being supplied to the render call. This way a user can make up any values they want/need and build a config template that matches their environment for any scenario but python-react can still use on the fly for specific components. The user can specify exactly how those components are rendered and which values need to come from python-react via the template key replacements and just supply them during the render call or even provide a dict of functions that computes them based on some application or state specific parameters (route name etc...).

For example, when I went to use python-webpack directly, it made sense to point it to a config file that I had already written in .js because if I wanted, I could supply a template config and dynamically inject the name of the component I wanted to render. I can't do that with python-react directly though. I would have to intercept it's usage of python-webpack or hard-code the template into python-react in replacement of the one that's currently used. (which I did for the time being)

The use case translates to python-react in a couple scenarios:

1) User has an existing webpack.config.js file that contains a whole bunch of external / library / output / plugin / loader rules that are constant for the entire project but I still want to render components on demand. I could supply a template of this config file to python-webpack specifying replacements in the entry fields for the component being rendered but the rest of the config is left untouched.

2) User has a production webpack.config.js separate from dev OR I have a webpack.generate.config.js that builds webpack config files based on certain conditions. I feed one or all of them into python-react for dynamic value injection (for components) and get consistent packing behavior across all environments without having to make python-webpack aware of the changing environmental concerns.

I ended up basically replacing the config template that python-react uses (for now) and it was having to do that made me decide that maybe that responsibility should not live inside the python-react package but rather be on the user to supply and pass through to python-webpack.

As far as testing this route goes, you would really only need to test that value injection was occuring. You would no longer need to test permutations of templates because that's now the responsibility of the user to supply a valid config template. You only need test that the value replacement happens correctly-- simple I should think.

I imagine the value replacement would also really be the responsibility of python-webpack. python-react would really just pass through.

If I've overlooked something obvious that precludes these ideas than I apologize, I would like to spend more time familiarizing myself with all the relevant code-bases anyway.

[markfinger]

That's a really good idea. The entry prop's all you'd need to override.

webpack(
    '/path/to/webpack.config.js',
    entry='/path/to/component.js',
)

[markfinger]

This would need support for entry overrides in https://github.com/markfinger/webpack-build and python-webpack.

Also, I've been musing over using config files which export a factory function. It would make it easier to make reusable template files and would avoid a lot of issues caused by JS's mutable objects.

[markfinger]

You could pass context down into the config factory, eg:

webpack(
    '/path/to/webpack.config.js',
    context={
        'entry': '/path/to/component.js',
        'DEBUG': false
    }
)

and in your config file

module.exports = function(opts) {
  var config = {
    entry: opts.context.entry
     // ...
  };

  if (opts.hmr) {
    // inject react hot loader
  }

  if (opts.context.DEBUG) {
    config.devtool = '...';
    // ...
  } else {
    // ...
  }

  return config;
};

[txbm]

Yes this is exactly what I had in mind. I may fork and start working on this because I need it right now. :)

Glad you like it.

[markfinger]

Cool.

Dev branch on python-webpack's diverged a fair bit from master. It uses webpack-build's server - rather than js-host - and there are a bunch of backwards incompatible changes in settings as well. Not sure how well python-react would talk to it currently, but it'd probably be just a bunch of renames.

To support something like this, you'd need:

• add a check to support factory functions at https://github.com/markfinger/webpack-build/blob/389960c15917b1cd4d86a8aea08dabaa1ed46ab1/src/wrappers/Wrapper.js#L51, something like if (_.isFunction(this.config)) { this.config = this.config(this.opts); }
• change webpack.compiler.webpack to accept extra args and to add those to the object sent to the server

[txbm]

Excellent. I'll likely have time to get into it more this weekend. This will really help with setting up config's that can resolve components from other module paths as well.

[markfinger]

This unblocked a bunch of pipeline stuff that I'd been musing over for a while, so I went ahead and handled the webpack side of things.

webpack-build@0.11 and python-webpack's dev branch have support for both config factories and context to be passed between the processes. Docs have been updated for both.

It's going to be a while until I get some time to tend to python-react, so you're more than welcome to open PRs with whatever works for you.

Cheers, Mark

[markfinger]

Latest release has stripped out the built-in webpack integration.

python-webpack's examples illustrate pushing data down to the config layer.

The self-mounting components example illustrates how to use python-webpack to define imports and variables.

[txbm]

Cool! Sorry I never got a chance to help out. Had to settle with what was working on account of deadlines. You seem to have made the separation perfectly though and I'll be using this new flow for the next refactor of my project. Thanks :)