search

apostrophecms / stagecoach

Painlessly deploy node.js applications to your staging and production servers. Use a standard VPS or dedicated server to host both Node and traditional Apache-based websites. Pairs nicely with nginx and mechanic.
306 stars 29 forks source link

Use of "npm install" on the server side is considered a bad practice #14

Open boutell opened 10 years ago

boutell commented 10 years ago

See:

https://github.com/isaacs/npm/issues/4304

I complained that npm publish does not have read-after-write consistency so I can't script doing a deploy after a publish of a public npm module and get reliable results.

The response was that I shouldn't be doing that anyway. Both for that reason and to avoid slamming npm's servers.

The proper practice would be to copy node_modules after all, as we did in the dawn days of stagecoach - then do a recursive "npm rebuild" on all the modules, to address binary incompatibility issues that caused us to use "npm install" remotely in the first place.

coreytegeler commented 8 years ago

I know this is old, but is this still the case?

I get errors from npm install and it might be in my favor to try and copy the node_modules folder or manually install them so I can work through errors rather than having the whole deployment stop and deleted

boutell commented 8 years ago

Well, the difficulties are mostly human and a little bit technical:

  1. We maintain a lot of modules which are npm linked in our projects. If deploying the app deploys the modules in their current hacked-up state, we may never commit and publish them responsibly (:

Forcing the use of npm install on the server side is one way to ensure that code in our own source projects gets committed, pushed and published properly if we want to use it in production.

  1. Deploying "npm link"ed modules by copying them is pretty challenging, even if it were a good idea. We'd have to configure rsync to copy through the symbolic links. Probably a better idea to detect whether npm link is in use and force the use of npm install if it is... but now sc-deploy has to check for this issue first... yuck.
  2. You can't, in fact, just copy over your node_modules folder in all cases. npm modules can and often do have C++ compiled components which are platform specific. Sometimes it's just a performance thing, other times you're actually out of luck without them.

However, you can in principle copy over your node_modules folder and then do "npm rebuild" to recompile anything platform-specific.

So right now our recommendation is to make sure you have enough RAM for npm to do its job. (Yes, it's a real pig.)

On Wed, Apr 20, 2016 at 12:34 PM, Corey Tegeler notifications@github.com wrote:

I know this is old, but is this still the case?

I get errors from npm install and it might be in my favor to try and copy the node_modules folder or manually install them so I can work through errors rather than having the whole deployment stop and deleted

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/punkave/stagecoach/issues/14#issuecomment-212503709

THOMAS BOUTELL, DEV & OPS P'UNK AVENUE | (215) 755-1330 | punkave.com

boutell commented 8 years ago

However... nobody's stopping you from trying this in your project:

  1. Change rsync_exclude.txt so that node_modules gets copied.
  2. Edit deployment/dependencies to do "npm rebuild" rather than "npm install".

If you don't have any "npm link"ed modules, it ought to work, and I'd be curious to hear about your experiences with it.

(Modifying these files is something you're intended to do. The deployment/ scripts are meant to get changed as needed to support individual projects.)

On Wed, Apr 20, 2016 at 12:42 PM, Tom Boutell tom@punkave.com wrote:

Well, the difficulties are mostly human and a little bit technical:

  1. We maintain a lot of modules which are npm linked in our projects. If deploying the app deploys the modules in their current hacked-up state, we may never commit and publish them responsibly (:

Forcing the use of npm install on the server side is one way to ensure that code in our own source projects gets committed, pushed and published properly if we want to use it in production.

  1. Deploying "npm link"ed modules by copying them is pretty challenging, even if it were a good idea. We'd have to configure rsync to copy through the symbolic links. Probably a better idea to detect whether npm link is in use and force the use of npm install if it is... but now sc-deploy has to check for this issue first... yuck.
  2. You can't, in fact, just copy over your node_modules folder in all cases. npm modules can and often do have C++ compiled components which are platform specific. Sometimes it's just a performance thing, other times you're actually out of luck without them.

However, you can in principle copy over your node_modules folder and then do "npm rebuild" to recompile anything platform-specific.

So right now our recommendation is to make sure you have enough RAM for npm to do its job. (Yes, it's a real pig.)

On Wed, Apr 20, 2016 at 12:34 PM, Corey Tegeler notifications@github.com wrote:

I know this is old, but is this still the case?

I get errors from npm install and it might be in my favor to try and copy the node_modules folder or manually install them so I can work through errors rather than having the whole deployment stop and deleted

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/punkave/stagecoach/issues/14#issuecomment-212503709

THOMAS BOUTELL, DEV & OPS P'UNK AVENUE | (215) 755-1330 | punkave.com

THOMAS BOUTELL, DEV & OPS P'UNK AVENUE | (215) 755-1330 | punkave.com

coreytegeler commented 8 years ago

Wonderful, this let me get passed the initial deployment very painfully.

I had done each step manually to make sure it worked before editing the scripts, but it's great to be able to customize those!

Thanks for the guide

boutell commented 8 years ago

Glad to hear this alternative does work! We will keep considering the best way to handle it in our own work.

On Wed, Apr 20, 2016 at 1:08 PM, Corey Tegeler notifications@github.com wrote:

Wonderful, this let me get passed the initial deployment very painfully.

I had done each step manually to make sure it worked before editing the scripts, but it's great to be able to customize those!

Thanks for the guide

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/punkave/stagecoach/issues/14#issuecomment-212517727

THOMAS BOUTELL, DEV & OPS P'UNK AVENUE | (215) 755-1330 | punkave.com