felixrieseberg / ember-cli-azure-deploy

:wrench: Build Ember Cli Apps on Azure Websites
MIT License
19 stars 2 forks source link

Deployments will sometimes fail on NPM #6

Closed cgsawtell closed 8 years ago

cgsawtell commented 9 years ago

Occasionally Azure will fail and deploying an ember project on npm looks like it might be a lock file issue??

Build log: Command: bash deploy.sh Copy assets to D:\local\Temp\38ca8421-9840-4704-a800-3a3f856c3000 for build ./ ./.bowerrc ./.deployment ./.editorconfig ./.ember-cli ./.gitignore ./.jshintrc ./.travis.yml ./app/ ./app/app.js ./app/components/ ./app/components/.gitkeep ./app/controllers/ ./app/controllers/.gitkeep ./app/helpers/ ./app/helpers/.gitkeep ./app/index.html ./app/models/ ./app/models/.gitkeep ./app/pods/ ./app/pods/pipeline/ ./app/pods/pipeline/controller.js ./app/pods/pipeline/model.js ./app/pods/pipeline/route.js ./app/pods/pipeline/template.hbs ./app/pods/pipeline-object/ ./app/pods/pipeline-object/template.hbs ./app/pods/pipeline-object/view.js ./app/pods/pipeline-object-handle/ ./app/pods/pipeline-object-handle/view.js ./app/pods/pipeline-track/ ./app/pods/pipeline-track/template.hbs ./app/pods/pipeline-track/view.js ./app/router.js ./app/routes/ ./app/routes/.gitkeep ./app/styles/ ./app/styles/app.css ./app/templates/ ./app/templates/application.hbs ./app/templates/components/ ./app/templates/components/.gitkeep ./app/templates/pipeline-client.hbs ./app/views/ ./app/views/.gitkeep ./bower.json ./Brocfile.js ./config/ ./config/environment.js ./deploy.sh ./package.json ./public/ ./public/crossdomain.xml ./public/robots.txt ./README.md ./testem.json ./tests/ ./tests/.jshintrc ./tests/helpers/ ./tests/helpers/resolver.js ./tests/helpers/start-app.js ./tests/index.html ./tests/test-helper.js ./tests/unit/ ./tests/unit/.gitkeep ./tests/unit/pods/ ./tests/unit/pods/pipeline/ ./tests/unit/pods/pipeline/controller-test.js ./tests/unit/pods/pipeline/model-test.js ./tests/unit/pods/pipeline/route-test.js ./tests/unit/pods/pipeline-object/ ./tests/unit/pods/pipeline-object/view-test.js ./tests/unit/pods/pipeline-track/ ./tests/unit/pods/pipeline-track/view-test.js ./tests/unit/pods/pipline-object-handle/ ./tests/unit/pods/pipline-object-handle/view-test.js ./vendor/ ./vendor/.gitkeep Switch to the temp directory ember-cli already installed, nothing to do bower already installed, nothing to do ember-cli-azure-deploy already installed, nothing to do Installing npm modules

ws@0.5.0 install D:\local\Temp\38ca8421-9840-4704-a800-3a3f856c3000\node_modules\ember-cli\node_modules\testem\node_modules\socket.io\node_modules\engine.io\node_modules\ws (node-gyp rebuild 2> builderror.log) || (exit 0) D:\local\Temp\38ca8421-9840-4704-a800-3a3f856c3000\node_modules\ember-cli\node_modules\testem\node_modules\socket.io\node_modules\engine.io\node_modules\ws>node "c:\Program Files (x86)\npm\1.4.28\node_modules\npm\bin\node-gyp-bin....\node_modules\node-gyp\bin\node-gyp.js" rebuild Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch. MSBUILD : error MSB3428: Could not load the Visual C++ component "VCBuild.exe". To fix this, 1) install the .NET Framework 2.0 SDK, 2) install Microsoft Visual Studio 2005 or 3) add the location of the component to the system path if it is installed elsewhere. [D:\local\Temp\38ca8421-9840-4704-a800-3a3f856c3000\node_modules\ember-cli\node_modules\testem\node_modules\socket.io\node_modules\engine.io\node_modules\ws\build\binding.sln] MSBUILD : error MSB3428: Could not load the Visual C++ component "VCBuild.exe". To fix this, 1) install the .NET Framework 2.0 SDK, 2) install Microsoft Visual Studio 2005 or 3) add the location of the component to the system path if it is installed elsewhere. [D:\local\Temp\38ca8421-9840-4704-a800-3a3f856c3000\node_modules\ember-cli\node_modules\testem\node_modules\socket.io\node_modules\engine.io\node_modules\ws\build\binding.sln] ws@0.4.31 install D:\local\Temp\38ca8421-9840-4704-a800-3a3f856c3000\node_modules\ember-cli\node_modules\testem\node_modules\socket.io\node_modules\socket.io-client\node_modules\engine.io-client\node_modules\ws (node-gyp rebuild 2> builderror.log) || (exit 0) D:\local\Temp\38ca8421-9840-4704-a800-3a3f856c3000\node_modules\ember-cli\node_modules\testem\node_modules\socket.io\node_modules\socket.io-client\node_modules\engine.io-client\node_modules\ws>node "c:\Program Files (x86)\npm\1.4.28\node_modules\npm\bin\node-gyp-bin....\node_modules\node-gyp\bin\node-gyp.js" rebuild Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch. MSBUILD : error MSB3428: Could not load the Visual C++ component "VCBuild.exe". To fix this, 1) install the .NET Framework 2.0 SDK, 2) install Microsoft Visual Studio 2005 or 3) add the location of the component to the system path if it is installed elsewhere. [D:\local\Temp\38ca8421-9840-4704-a800-3a3f856c3000\node_modules\ember-cli\node_modules\testem\node_modules\socket.io\node_modules\socket.io-client\node_modules\engine.io-client\node_modules\ws\build\binding.sln] MSBUILD : error MSB3428: Could not load the Visual C++ component "VCBuild.exe". To fix this, 1) install the .NET Framework 2.0 SDK, 2) install Microsoft Visual Studio 2005 or 3) add the location of the component to the system path if it is installed elsewhere. [D:\local\Temp\38ca8421-9840-4704-a800-3a3f856c3000\node_modules\ember-cli\node_modules\testem\node_modules\socket.io\node_modules\socket.io-client\node_modules\engine.io-client\node_modules\ws\build\binding.sln] npm ERR! Error: EPERM, open 'D:\local\AppData\npm-cache\591337a6-race-expansion-1-1-0-package-tgz.lock' npm ERR! { [Error: EPERM, open 'D:\local\AppData\npm-cache\591337a6-race-expansion-1-1-0-package-tgz.lock'] npm ERR! errno: 50, npm ERR! code: 'EPERM', npm ERR! path: 'D:\local\AppData\npm-cache\591337a6-race-expansion-1-1-0-package-tgz.lock' } npm ERR! npm ERR! Please try running this command again as root/Administrator. npm ERR! System Windows_NT 6.2.9200 npm ERR! command "d:\Program Files (x86)\nodejs\0.10.32\node.exe" "D:\Program Files (x86)\npm\1.4.28\node_modules\npm\bin\npm-cli.js" "install" "--no-optional" "--no-bin-links" npm ERR! cwd D:\local\Temp\38ca8421-9840-4704-a800-3a3f856c3000 npm ERR! node -v v0.10.32 npm ERR! npm -v 1.4.28 npm ERR! path D:\local\AppData\npm-cache\591337a6-race-expansion-1-1-0-package-tgz.lock npm ERR! code EPERM npm ERR! errno 50 npm ERR! stack Error: EPERM, open 'D:\local\AppData\npm-cache\591337a6-race-expansion-1-1-0-package-tgz.lock' An error has occurred during web site deployment. npm install failed npm

felixrieseberg commented 9 years ago

Hey, thanks for the report. It seems like the npm cache is what's making trouble here - I'll play around, I believe that we could be just fine without the cache.

joukevandermaas commented 9 years ago

I get a different error, but it seems like it has a similar cause:

bower already installed, nothing to do
Using Node Using npm 1.4.28
Using bower Using ember-cli-azure-deploy 1.2.5
Cleaning Cache
Installing npm modules

> node-sass@3.2.0 install D:\local\Temp\9cd93aa0-25e0-4605-82eb-746f7fb93b2c\node_modules\ember-cli-sass\node_modules\broccoli-sass-source-maps\node_modules\node-sass
> node scripts/install.js

Binary downloaded and installed at D:\local\Temp\9cd93aa0-25e0-4605-82eb-746f7fb93b2c\node_modules\ember-cli-sass\node_modules\broccoli-sass-source-maps\node_modules\node-sass\vendor\win32-ia32-11\binding.node
npm ERR! EEXIST, open 'D:\local\AppData\npm-cache\9e2a2bbc-pm-cache-rsvp-3-0-18-package-tgz.lock'

File exists: D:\local\AppData\npm-cache\9e2a2bbc-pm-cache-rsvp-3-0-18-package-tgz.lock
> node-sass@3.2.0 postinstall D:\local\Temp\9cd93aa0-25e0-4605-82eb-746f7fb93b2c\node_modules\ember-cli-sass\node_modules\broccoli-sass-source-maps\node_modules\node-sass
Move it away, and try again. 
> node scripts/build.js

npm ERR! System Windows_NT 6.2.9200
` D:\local\Temp\9cd93aa0-25e0-4605-82eb-746f7fb93b2c\node_modules\ember-cli-sass\node_modules\broccoli-sass-source-maps\node_modules\node-sass\vendor\win32-ia32-11\binding.node ` exists. 
npm ERR! command "d:\\Program Files (x86)\\nodejs\\0.10.32\\node.exe" "D:\\Program Files (x86)\\npm\\1.4.28\\node_modules\\npm\\bin\\npm-cli.js" "install" "--no-optional" "--no-bin-links"
testing binary.
npm ERR! cwd D:\local\Temp\9cd93aa0-25e0-4605-82eb-746f7fb93b2c
Binary is fine; exiting.
npm ERR! node -v v0.10.32
npm ERR! npm -v 1.4.28
npm ERR! path D:\local\AppData\npm-cache\9e2a2bbc-pm-cache-rsvp-3-0-18-package-tgz.lock
npm ERR! code EEXIST
npm ERR! errno 47
An error has occurred during web site deployment.
npm install failed

This happens on every build, so I can't deploy my project using this tool currently.

felixrieseberg commented 9 years ago

Hm, I don't really know what to do about that, since we're clearing the cache right before installing. I'll involve Kudu on this one.

EDIT: On second thought, I'm not sure that this is a Kudu issue.

felixrieseberg commented 9 years ago

@joukevandermaas: Let's try a newer version of npm here - npm fixed a bunch of race conditions and other lock-file-related issues in later versions, so I'm wondering if the following will fix your issue.

After running azure-deploy init in your project, it creates a deploy.sh file. Do me a favor, open up the file and change line 50 from:

NPM_CMD="\"$NODE_EXE\" \"$PROGRAMFILES\\npm\\1.4.28\\node_modules\\npm\\bin\\npm-cli.js\""

to

NPM_CMD="\"$NODE_EXE\" \"$PROGRAMFILES\\npm\\2.9.1\\node_modules\\npm\\bin\\npm-cli.js\""

I'm wondering if a newer npm version will fix your issues.

joukevandermaas commented 9 years ago

It still fails (i tried about 5 times, so it is consistent), but with a different error:

> ws@0.5.0 install D:\local\Temp\ceb3142a-005d-46ea-81df-a059242bde11\node_modules\socket.io\node_modules\engine.io\node_modules\ws
> (node-gyp rebuild 2> builderror.log) || (exit 0)

D:\local\Temp\ceb3142a-005d-46ea-81df-a059242bde11\node_modules\socket.io\node_modules\engine.io\node_modules\ws>if not defined npm_config_node_gyp (node "c:\Program Files (x86)\npm\2.9.1\node_modules\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (rebuild) 
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
MSBUILD : error MSB3428: Could not load the Visual C++ component "VCBuild.exe". To fix this, 1) install the .NET Framework 2.0 SDK, 2) install Microsoft Visual Studio 2005 or 3) add the location of the component to the system path if it is installed elsewhere.  [D:\local\Temp\ceb3142a-005d-46ea-81df-a059242bde11\node_modules\socket.io\node_modules\engine.io\node_modules\ws\build\binding.sln]
MSBUILD : error MSB3428: Could not load the Visual C++ component "VCBuild.exe". To fix this, 1) install the .NET Framework 2.0 SDK, 2) install Microsoft Visual Studio 2005 or 3) add the location of the component to the system path if it is installed elsewhere.  [D:\local\Temp\ceb3142a-005d-46ea-81df-a059242bde11\node_modules\socket.io\node_modules\engine.io\node_modules\ws\build\binding.sln]
socket.io@1.3.5 node_modules\socket.io
+-- has-binary-data@0.1.3 (isarray@0.0.1)
+-- debug@2.1.0 (ms@0.6.2)
+-- socket.io-client@1.3.5 (to-array@0.1.3, indexof@0.0.1, debug@0.7.4, object-component@0.0.3, component-bind@1.0.0, component-emitter@1.1.2, backo2@1.0.2, has-binary@0.1.6, parseuri@0.0.2)
+-- socket.io-adapter@0.3.1 (object-keys@1.0.1, debug@1.0.2, socket.io-parser@2.2.2)
+-- socket.io-parser@2.2.4 (isarray@0.0.1, debug@0.7.4, component-emitter@1.1.2, benchmark@1.0.0, json3@3.2.6)
+-- engine.io@1.5.1 (base64id@0.1.0, debug@1.0.3, engine.io-parser@1.2.1, ws@0.5.0)
testem@0.8.3 node_modules\testem
+-- growl@1.8.1
+-- styled_string@0.0.1
+-- did_it_work@0.0.6
+-- consolidate@0.12.1
+-- charm@1.0.0
+-- colors@1.1.2
+-- xmldom@0.1.19
+-- async@0.9.2
+-- mustache@2.1.2
+-- commander@2.8.1 (graceful-readlink@1.0.1)
+-- cross-spawn@0.2.9 (lru-cache@2.6.4)
+-- http-proxy@1.11.1 (eventemitter3@1.1.1, requires-port@0.0.1)
+-- mkdirp@0.5.1 (minimist@0.0.8)
+-- backbone@1.2.1 (underscore@1.8.3)
+-- glob@5.0.10 (path-is-absolute@1.0.0, inherits@2.0.1, once@1.3.2, inflight@1.0.4, minimatch@2.0.8)
+-- rimraf@2.4.0 (glob@4.5.3)
+-- npmlog@1.2.1 (ansi@0.3.0, gauge@1.2.0, are-we-there-yet@1.0.4)
+-- fileset@0.1.8 (minimatch@0.4.0, glob@3.2.11)
+-- express@4.13.0 (escape-html@1.0.2, utils-merge@1.0.0, merge-descriptors@1.0.0, array-flatten@1.1.0, cookie@0.1.3, content-type@1.0.1, parseurl@1.3.0, methods@1.1.1, cookie-signature@1.0.6, fresh@0.3.0, range-parser@1.0.2, path-to-regexp@0.1.6, vary@1.0.0, etag@1.7.0, serve-static@1.10.0, content-disposition@0.5.0, depd@1.0.1, finalhandler@0.4.0, on-finished@2.3.0, qs@2.4.2, debug@2.2.0, proxy-addr@1.0.8, send@0.13.0, type-is@1.6.3, accepts@1.2.9)
+-- fireworm@0.6.6 (async@0.2.10, is-type@0.0.1, minimatch@0.2.14, lodash@2.3.0)
+-- tap@0.7.1 (inherits@2.0.1, buffer-equal@0.0.1, deep-equal@1.0.0, slide@1.1.6, yamlish@0.0.7, nopt@3.0.3, difflet@0.2.6, glob@4.5.3, runforcover@0.0.2)
+-- js-yaml@3.3.1 (esprima@2.2.0, argparse@1.0.2)
Installing ember-cli
npm ERR! Windows_NT 6.2.9200
npm ERR! argv "d:\\Program Files (x86)\\nodejs\\0.10.32\\node.exe" "D:\\Program Files (x86)\\npm\\2.9.1\\node_modules\\npm\\bin\\npm-cli.js" "install" "--no-optional" "--no-bin-links" "ember-cli"
npm ERR! node v0.10.32
npm ERR! npm  v2.9.1
npm ERR! path D:\local\Temp\ceb3142a-005d-46ea-81df-a059242bde11\node_modules\ember-cli\node_modules\broccoli-sourcemap-concat\node_modules\fast-sourcemap-concat\node_modules\chalk\node_modules\strip-ansi\node_modules\ansi-regex\package.json.372c6136f235c6ad93a683d054298fd4
npm ERR! code EINVAL
npm ERR! errno 18

npm ERR! EINVAL, rename 'D:\local\Temp\ceb3142a-005d-46ea-81df-a059242bde11\node_modules\ember-cli\node_modules\broccoli-sourcemap-concat\node_modules\fast-sourcemap-concat\node_modules\chalk\node_modules\strip-ansi\node_modules\ansi-regex\package.json.372c6136f235c6ad93a683d054298fd4'
npm ERR! 
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>

An error has occurred during web site deployment.
ember-cli failed

Which seems a lot more cryptic. Maybe the lock file issue is fixed in this release?

gitowiec commented 9 years ago

Inside Brocfile.js on the project level enter

var app = new EmberApp({
    minifyJS: {
        enabled: false
    },
    sourcemaps: {
        enabled: false,
        extensions: ['js']
    }
});

It helped me with sourcemap errors

npm ERR! EINVAL, rename 'D:\local\Temp\ceb3142a-005d-46ea-81df-a059242bde11\node_modules\ember-cli\node_modules\broccoli-sourcemap-concat\node_modules\fast-sourcemap-concat\node_modules\chalk\node_modules\strip-ansi\node_modules\ansi-regex\package.json.372c6136f235c6ad93a683d054298fd4'

Additionaly if you are attaching ready (bought) skin (html template) - just get rid of all unused JavaScript plugins that you are not sure of. Deploying with -prod makes deep check of all sources.

felixrieseberg commented 9 years ago

We just deployed npm@3 to Azure Websites, which will hopefully fix a bunch of lock file issues. I'll update the script to use it, let's see how much luck we all have then :rabbit:

joukevandermaas commented 9 years ago

Sorry for the late reaction. That update would be great. For now we have fallen back to manual deployment (without kudu).

felixrieseberg commented 9 years ago

@joukevandermaas Thanks! You could actually help me out a lot if you could see if npm@3 fixes your issue - because we have also seen a few people struggle with Ember Cli and npm@3 (which has no connection with Azure, but if Ember doesn't like npm@3 yet, we won't have too much fun with it on Azure, either). Could you try installing your project with it (if it works on your machine, we can probably make it work on Azure, too)? I'd be super thankful :smiley:

joukevandermaas commented 9 years ago

@felixrieseberg I just tried this and it would not work. It seems that ember cli makes assumptions about the directory structure of node_modules. I'll try investigate some further into what is doing this and why, but it seems like this won't be a quick fix (for our project). I also tried this for a new ember project, and that did work, so it's definitely one of our dependencies. We'll investigate some more on what is breaking and whether we actually need that thing.

I only tried on Linux so far, but I'll see if I can give it a try on Windows sometime later this week. I doubt it'll make much difference, though. I couldn't really find any npm related issues on the ember cli repository, so maybe I should push them to look at npm3 some more as well.

joukevandermaas commented 8 years ago

Small (and very late, sorry) update: npm@3 does actually solve the issue where npm install fails during the deployment, but then the ember build fails afterwards. From what I can tell this occurs because Broccoli somewhere assumes some addon will have a node_modules subdir. I will investigate more and possibly open an issue in their repo.

FYI, here's the relevant part of the log:

Executing "ember build -prod". This might take a while.
Future versions of Ember CLI will not support v0.10.32. Please update to Node 0.12 or io.js.
version: 1.13.1
BuildingBuilding.Building..Build failed.
ENOENT, no such file or directory 'D:\local\Temp\207352e4-c129-41a5-afe2-cb8017d2044a\node_modules\ember-cli-ic-ajax\node_modules'
Error: ENOENT, no such file or directory 'D:\local\Temp\207352e4-c129-41a5-afe2-cb8017d2044a\node_modules\ember-cli-ic-ajax\node_modules'
    at Object.fs.readdirSync (fs.js:665:18)
    at D:\local\Temp\207352e4-c129-41a5-afe2-cb8017d2044a\node_modules\broccoli-merge-trees\index.js:32:21
    at Array.map (native)
    at mergeRelativePath (D:\local\Temp\207352e4-c129-41a5-afe2-cb8017d2044a\node_modules\broccoli-merge-trees\index.js:30:29)
    at D:\local\Temp\207352e4-c129-41a5-afe2-cb8017d2044a\node_modules\broccoli-merge-trees\index.js:23:5
    at lib$rsvp$$internal$$tryCatch (D:\local\Temp\207352e4-c129-41a5-afe2-cb8017d2044a\node_modules\rsvp\dist\rsvp.js:493:16)
    at lib$rsvp$$internal$$invokeCallback (D:\local\Temp\207352e4-c129-41a5-afe2-cb8017d2044a\node_modules\rsvp\dist\rsvp.js:505:17)
    at lib$rsvp$$internal$$publish (D:\local\Temp\207352e4-c129-41a5-afe2-cb8017d2044a\node_modules\rsvp\dist\rsvp.js:476:11)
    at Object.lib$rsvp$asap$$flush [as _onImmediate] (D:\local\Temp\207352e4-c129-41a5-afe2-cb8017d2044a\node_modules\rsvp\dist\rsvp.js:1198:9)
    at processImmediate [as _immediateCallback] (timers.js:345:15)
Copy web.config to the dist folder
KuduSync.NET from: 'D:\local\Temp\207352e4-c129-41a5-afe2-cb8017d2044a\dist' to: 'D:\home\site\wwwroot'
An error has occurred during web site deployment.
Kudu Sync failed

As you can see this occurs with one of the default addons, and one that we absolutely depend on. I don't expect you to do anything about this of course, since this appears to be an ember-cli issue.

felixrieseberg commented 8 years ago

The good news: My colleague and friend @tylergibson already fixed the issue (https://github.com/ember-cli/ember-cli-preprocessor-registry/pull/3), we're just waiting for a new release of Ember Cli :+1:

mattmazzola commented 8 years ago

I'm having the same error as joukevandermaas. When the deploy script is attempting to install node-sass it fails.

Based on previous comments it seems like a solution is known and we're just waiting. Are these the steps that we're expecting?

  1. ember-cli will push out new release (1.13.8) which includes the fix mentioned above which allows it to be used with npm 3.
  2. deploy.sh will be updated to use npm 3.
  3. We all upgrade our ember apps to use the new ember-cli
  4. Update our global ember-cli-deploy-azure
  5. Run azure-deploy init on the ember app root directory again to add the updated deploy.sh
felixrieseberg commented 8 years ago

Yep, that sounds about right. @stefanpenner just released another version, so I'll update this to have an npm3 option tomorrow.

felixrieseberg commented 8 years ago

Thank you very much to everyone - just published ember-cli-azure-deploy 1.3.0, which uses npm3. I hope that it'll solve most of the issues, so I'll be closing this for now.

stefanpenner commented 8 years ago

;)