csvan
commented
7 years ago see https://github.com/angular-fullstack/generator-angular-fullstack/issues/2342 for some workarounds.
Open stephengardner opened 7 years ago
csvan
commented
7 years ago see https://github.com/angular-fullstack/generator-angular-fullstack/issues/2342 for some workarounds.
stephengardner
commented
7 years ago I'm new to Webpack so I'm genuinely confused by this being OK.
Don't people make server edits and have the changes reload fast? My client edits are fast after making additional tweaks, but the server edits are atrocious.
They weren't this way with the old generator.
Webpack is recreating everything for any minimal update on any server file? I'm just confused how the fullstack generator team had builds far faster before, and regressed to a slow build speed? Adding webpack is great, but is this speed literally the minimal?
csvan
commented
7 years ago @stephengardner One theory is that since changing a server file triggers a nodemon restart, this in turn triggers an entire clean webpack build. Contrast this with changing a client file which makes use of the standard webpack caching features.
You can verify this by comparing the initial webpack build time when running gulp serve to the time it takes to refresh after changing a server file. They are virtually the same on a new project (7-8s on my box).
I suspect the problem lies somewhere in the Webpack middleware for express, though I am not too clear on how it works.
augiegardner
commented
7 years ago @csvan that general idea is certainly THE issue, not just speculation. The full build refresh IS The problem.
My client builds I have gotten down to something like a single second time to full refresh (on a large project), however, the server refreshes are killing me.
If I was new to the fullstack generator I suppose this would be okay, but the earlier versions were at LEAST twice as fast.
I'm just surprised no one else has raised this issue... This is certainly a big deal, unless I'm completely missing something
leebailey88
commented
7 years ago My client edits are 1-2seconds, but my server edits are ~14s which is practically unworkable. Agree with all of those who said they miss the old version of the generator where server changes were much faster.
augiegardner
commented
7 years ago Yep, my server edits take 11-12 seconds to rebuild plus the general server reload time. Ends up something like 15 seconds overall. Client edits: blazing fast.
Hopefully someone can chime in. Again it's easy to test just doing a fresh scaffolded app, but I have tried fruitlessly to really get this reasonable.
I spent many hours refactoring my old environment to use this new one. Definitely surprising.
On Nov 20, 2016 9:33 PM, "Lee Bailey" notifications@github.com wrote:
My client edits are 1-2seconds, but my server edits are ~14s which is practically unworkable. Agree with all of those who said they miss the old version of the generator where server changes were much faster.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/angular-fullstack/generator-angular-fullstack/issues/2352#issuecomment-261828478, or mute the thread https://github.com/notifications/unsubscribe-auth/AFUa6UnxMBJDz9z_o4GsY84TZJJDtBNFks5rAQLigaJpZM4K08qw .
csvan
commented
7 years ago The solution to this (I think) would be to run the Webpack dev server and Express backend as separate instances. server/config/express.js is reloaded every time Nodemon resets the server due to changes, causing the entire Webpack instance to be restarted as well. I'm not sure it is even possible to work around this as it is configured now.
augiegardner
commented
7 years ago Interesting... Would you happen to have an idea of the code required to set up your hypothetical environment?
On Mon, Nov 21, 2016 at 12:57 AM, Christopher Svanefalk < notifications@github.com> wrote:
The solution to this (I think) would be to run the Webpack dev server and Express backend as separate instances. server/config/express.js is reloaded every time Nodemon resets the server due to changes, causing the entire Webpack instance to be restarted as well. I'm not sure it is even possible to work around this as it is configured now.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/angular-fullstack/generator-angular-fullstack/issues/2352#issuecomment-261850545, or mute the thread https://github.com/notifications/unsubscribe-auth/AFUa6bAGy6eSWHNsjwiYgE44HnDt-fYEks5rATKzgaJpZM4K08qw .
csvan
commented
7 years ago @augiegardner at work at the moment so I don't have too much time to dig into it. However, a good start would be this: https://webpack.github.io/docs/usage-with-gulp.html
You will most likely then need to fiddle a bit with the port bindings to make sure your app still talks to the Express backend when doing authentication etc.
stephengardner
commented
7 years ago I'll take a look.
Hopefully more people find this to be an issue and we formulate a solution faster than not.
leebailey88
commented
7 years ago As others have mentioned, the slowness is due to the webpack bundle being completely regenerated on ANY server change even tho your client side bundle hasn't actually changed at all. This is very wasteful and time consuming. The code that does this is inside server/config/express.js which is reloaded on any change thanks to nodemon. I believe this is also the reason client side updates are blazing fast due to the in-memory nature of webpack-dev-middleware.
This client side speed is coming with the crippling side effect of these incredibly slow server change incremental builds. At a high level, I see a few possible solutions:
require.cache cleaning? SIGUSR2 from (nodemon on file change) to somehow save the in-memory bundles generated by webpack-dev-middleware and then reload them from disk instead of regenerating them on the subsequent run of server/config/express.jsserver-config-express.js in a separate process so that it isn't restarted when server files changeI don't have time this week to look into the details of any of these ideas, but does anyone think one of these would be reasonable?
stephengardner
commented
7 years ago I like what you've mentioned @leebailey88, it seems like the last option might be the most straightforward way to accomplish this. Any repo admins want to weigh in?
leebailey88
commented
7 years ago For now I've just extracted out the webpack dev middleware stuff from server/config/express.js to a separate file:
const webpackDevMiddleware = require('webpack-dev-middleware');
const stripAnsi = require('strip-ansi');
const webpack = require('webpack');
const makeWebpackConfig = require('./webpack.make');
const webpackConfig = makeWebpackConfig({ DEV: true });
const compiler = webpack(webpackConfig);
const browserSync = require('browser-sync').create('foo');
/**
* Run Browsersync and use middleware for Hot Module Replacement
*/
browserSync.init({
open: false,
logFileChanges: false,
proxy: 'localhost:8999',
ws: true,
middleware: [
webpackDevMiddleware(compiler, {
noInfo: false,
stats: {
colors: true,
timings: true,
chunks: false,
},
}),
],
port: 3000,
plugins: ['bs-fullscreen-message'],
});
/**
* Reload all devices when bundle is complete
* or send a fullscreen error message to the browser instead
*/
compiler.plugin('done', (stats) => {
console.log('webpack done hook');
if (stats.hasErrors()/* || stats.hasWarnings()*/) {
return browserSync.sockets.emit('fullscreen:message', {
title: 'Webpack Error:',
body: stripAnsi(stats.toString()),
timeout: 100000,
});
}
browserSync.reload();
});I called this file webpack.js, stuck it in my project root and then run it with node webpack.js. I completely removed that code from the express.js file. I just run two terminal windows now, in my other one is the normal gulp serve command but it no longer does the webpack stuff, so server code iterations are fast again. Client iterations are still fast because that's handled by the separate process started with node webpack.js.
The only problem now is that the browser doesn't automatically refresh any more on server changes (it still does on client changes), so I have to manually refresh after the server restarts. It's still much quicker than rebuilding the whole bundle each time like before, but it would be nicer if we could somehow hook into the browsersync from the express.js file and tell it to reload, but I don't know how to connect to the already running browsersync from the other process. Any ideas? @stephengardner @Awk34 @csvan
stephengardner
commented
7 years ago Fan-tastic. I'll have a look at this likely tomorrow. Will have to read up on exactly how browser sync operates but perhaps in the ´gulp serve´ we can add a watch on all files in the the ´serverPath´ which hooks into a refresh.
stephengardner
commented
7 years ago On second thought I'm thinking that would refresh the page too fast, since it would ping immediately. Yeah I'll check some things out.
stephengardner
commented
7 years ago Thanks to @leebailey88 for helping out
Here's the general idea:
webpack-dev-serverShould take just a few minutes to set up, and is relatively straightforward.
Create a file in the project root called serve-webpack.js
const webpackDevMiddleware = require('webpack-dev-middleware');
const stripAnsi = require('strip-ansi');
const webpack = require('webpack');
const makeWebpackConfig = require('./webpack.make');
const webpackConfig = makeWebpackConfig({ DEV: true });
const compiler = webpack(webpackConfig);
const browserSync = require('browser-sync').create();
var config = require("./server/config/environment");
/**
* Run Browsersync and use middleware for Hot Module Replacement
*/
browserSync.init({
open: false,
logFileChanges: false,
proxy: `localhost:${config.port}`,
ws: true,
middleware: [
webpackDevMiddleware(compiler, {
noInfo: false,
stats: {
colors: true,
timings: true,
chunks: false,
},
}),
],
port: config.browserSyncPort,
plugins: ['bs-fullscreen-message'],
});
/**
* Reload all devices when bundle is complete
* or send a fullscreen error message to the browser instead
*/
compiler.plugin('done', (stats) => {
console.log('webpack done hook');
if (stats.hasErrors()/* || stats.hasWarnings()*/) {
return browserSync.sockets.emit('fullscreen:message', {
title: 'Webpack Error:',
body: stripAnsi(stats.toString()),
timeout: 100000,
});
}
browserSync.reload();
});Remove that exact code from server/config/express.js. Angular-fullstack comes with this code inherently, so you need to comment it out or remove it from that file.
Also in server/config/express.js, add this code within the development block.
(Within the block that begins with if ('development' === env || 'test' === env) {
// creating a random text file to trigger a page refresh after the server reloads.
let fs = require('fs');
let clientTextFile = path.join(__dirname, '../../../../client/random.txt');
fs.open(clientTextFile, 'wx', (err, data) => {
fs.writeFile(clientTextFile, Date.now(), (err) => {
if(err) console.log(err);
});
});In client/app.js add the following to the top of the file:
// create a dependency between app.js and our server-generated text file
let random = require('../random.txt');(If you want, you can pre-create that random.txt file so the compiler doesn't yell at you for requiring an unknown file)
open a terminal window
run node serve-webpack
open a separate terminal window
run gulp serve
Now server edits will refresh the server, without recompiling your client-side bundle. If you're familiar with the old Fullstack Generator, what this means is that your server edits are now just as fast as they used to be.
The server refresh will re-write a client side file with a new number every time it reloads. Since client/app.js is requireing this file, webpack will monitor this client side file for changes, and refresh the page if it changes. Thus triggering a client side refresh when your server is done reloading, and all goes well.
This is certainly a temp solution, and would be tweaked if you were using HMR.
But for now, it is shaving a good 15 seconds off of every server-side edit. And keeps me one step closer to maintaining my sanity.
Awk34
commented
7 years ago I've been investigating this: https://github.com/TND/webpack-server-runner-plugin
I've got a better solution, similar to the one @stephengardner presents above.. I'll try to post a guide soon
Awk34
commented
7 years ago Here's my solution: https://github.com/Awk34/afs-hmr-test/compare/72206ccbc9e255be1afa690967dcf3df5665609f...176c5c1ce5915036b98754897c1e41366c1dccd2. These are changes between a project generated using the canary branch of the generator with default settings.
You still just run gulp serve. The front-end now just uses webpack-dev-server for serving the client. It also includes hot module replacement (HMR). The webpack code has been removed from server/config/express.js.
I've observed client changes to take a few seconds to hot update (no browser refresh). I've noticed server changes to take a few seconds to take effect, in which my browser shows a WebSocket disconnect, and then continues as normal once the server is back up. It would go quite faster if you disabled re-seeding the database on any server change. I'm quite pleased with this solution.
This is based on the canary changes for Angular 2.x.x. You can ignore the HMR stuff if you want to integrate this with an Angular 1.6 project. I hope this can get put into the generator's 4.x.x version as well as 5.0.0.
stephengardner
commented
7 years ago @Awk34 the current version uses webpack-dev-server, too. I think the fix you provided could use a better explanation, similar to my explanation above. Considering not many people have commented here I'm imagining people are still looking for an easy straightforward fix. HMR requires new modules to be written in a way that accommodates the HMR spec. So, unless I'm wrong, implementing your fix if every module relies then on HMR would be a bit tough. Though the mods of this repo may benefit from your version, though. This is certainly still an outstanding issue without an ideal solution.
Awk34
commented
7 years ago The current version of the generator just uses webpack-dev-middleware in our own express server. The code I posted uses the server, which itself uses the dev middleware. This allows easy use of HMR with Angular 2. I'm not sure about HMR with Angular 1. We'll probably just stick to live reload for Angular 1. For Angular 2, though, you shouldn't need to change your main app code at all.
Try cloning my repo and trying out the solution for yourself
yoshiokatsuneo
commented
7 years ago Based on @stephengardner and @leebailey88 solution, I could solve the issue by changing like below.
After the editing, just "gulp serve" should work.
const webpackDevMiddleware = require('webpack-dev-middleware');
const stripAnsi = require('strip-ansi');
const webpack = require('webpack');
const makeWebpackConfig = require('./webpack.make');
const webpackConfig = makeWebpackConfig({ DEV: true });
const compiler = webpack(webpackConfig);
const browserSync = require('browser-sync').create();export default function(app) { var config = require("./server/config/environment");
/**
Run Browsersync and use middleware for Hot Module Replacement
*/
browserSync.init({
open: false,
logFileChanges: false,
proxy: localhost:${config.port},
ws: true,
middleware: [
webpackDevMiddleware(compiler, {
noInfo: false,
stats: {
colors: true,
timings: true,
chunks: false,
},
}),
],
port: config.browserSyncPort,
plugins: ['bs-fullscreen-message'],
});
/**
Open "sever/config/express.js" , and replace inside "if(env === 'development')" like below:
if(env === 'development'){
// Notify webpack to reload browser
let fs = require('fs');
let clientTextFile = path.join(__dirname, '../../client/touch.js');
fs.writeFile(clientTextFile, Date.now(), (err) => {
if(err) console.error(err);
});
}Add empty "client/touch.txt".
Edit "client/app/app.js" Add a line to watch "touch.txt".
import '../touch';Add line below:
import serveWebpack from './serve-webpack';
gulp.task('serve:webpack', () => {
process.env.NODE_ENV = process.env.NODE_ENV || 'development';
serveWebpack();
});Add the "serve:webpack" task to "serve" task.
gulp.task('serve', cb => {
runSequence(
[
'clean:tmp',
'lint:scripts',
'inject',
'copy:fonts:dev',
'env:all'
],
// 'webpack:dev',
['start:server', 'serve:webpack', 'start:client'],
'watch',
cb
);
});@yoshiokatsuneo I knew I'd be able to add that in the webpack build somehow! Thanks! tiny typo:
Add "server-webpack.js" on the top directory:
Change to:
Add "serve-webpack.js" on the top directory:
(later you call this file serve, not server)
yoshiokatsuneo
commented
7 years ago @stephengardner Thanks! I fixed the typo.
royeeshemesh
commented
7 years ago @yoshiokatsuneo @stephengardner you guys just made my day! Awesome!!
Simply running a fresh
yo angular-fullstackbuild and editing a server file calls the watch on nodemon and takes over 6 seconds consistently to reload. It's taking longer obviously with any more code. (In the output below, Note theTime:I have highlighted)Any tips on reducing this re-build time?
I was using the outdated non-webpack version of generator for a year or two, and the server re-loads were about twice this fast.
(My computer is an upgraded Dell XPS 15, so it's not the computer)
generator-angular-fullstack | 4.1.1 Node | 6.2.2 npm | 3.9.5 Operating System | Windows 10
Transpiler Options: TypeScript Socket.io: Y HTML SCSS ui-router Mocha MongoDB Auth: Y (all)