LearningLocker / deploy

Official Deployment Scripts for Learning Locker
https://learninglocker.net
9 stars 28 forks source link

Install fails: '/tmp/learninglocker/webapp/node_modules' doesn't exist #57

Closed jenseeckhout closed 5 years ago

jenseeckhout commented 5 years ago

I'm running into an error when trying to install LL v2 via the standard install script. Everything runs fine until the script starts copying over the files from /tmp to the actual destination directory. I get the error can't copy directory '/tmp/learninglocker/webapp/node_modules' as it doesn't exist- exiting. When I cd to the tmp folder, I can indeed see that there is no node_modules directory. install.log is attached below. Any help would be greatly appreciated, as this is kind of a burning issue for me.

mongod --version: db version v2.6.12 redis-server --version: Redis server v=3.2.12 nginx -v: nginx/1.12.2 OS: 3.10.0-862.14.4.el7.centos.plus.x86_64

NOTE :: Local install selected
[LL] We require a path to install to and a path to symlink to. The reason for this is that the script can be re-run in order to update
cleanly. The path we'll ask you for is a base path for the releases to be installed to so if you select the default of:
/usr/local/learninglocker/releases
then we will create a sub-directory under here for every release and symlink the latest install to the final install path (which
the nginx config points at. This is so that roll-backs can be done easier and we can perform a complete install before finally
switching the nginx config over which'll minimise downtime on upgrades
[LL] What base directory do you want to install to? (Press 'enter' for the default of /usr/local/learninglocker/releases)
attempting to use base path of: /usr/local/learninglocker/releases
[LL] What path should the release be symlinked to? (Press enter for the default of /usr/local/learninglocker/current)
attempting to use path of: /usr/local/learninglocker/current
[LL] I need a user to install the code under - what user would you like me to use? (press enter for the default of 'learninglocker')
[LL] User 'learninglocker' already exists - are you sure you want to continue? [y|n] (enter for default of 'y')
continuing using this user
[LL] MongoDB is already installed, not installing
mongo version currently installed:
mongo check is 1
[LL] Mongo version () is above minimum of 3.0.0 - continuing
[LL] Redis is already installed, not installing
Redis Version: 3.2.12
[LL] Redis version (3.2.12) is above minimum of 2.8.11 - continuing
[LL] Learning Locker ideally works best with ClamAV (anti virus software) installed but it is not an absolute requirement. Do you want to install it? [y|n] (press 'enter' for the default of 'y')
user entered ''
Detected OS: Redhat, subver:CentOS, arch:x86_64, vno:7, NodeOverride: false, PM2Override:false
[LL] Installing node version: 8.x
[LL] installing base software....
Failed to set locale, defaulting to C
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos.mirror.nucleus.be
 * centosplus: centos.mirror.nucleus.be
 * epel: epel.mirror.nucleus.be
 * extras: centos.mirror.nucleus.be
 * updates: centos.mirror.nucleus.be
http://repo.zabbix.com/zabbix/3.0/rhel//x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
To address this issue please refer to the below wiki article

https://wiki.centos.org/yum-errors

If above article doesn't help to resolve this issue please use https://bugs.centos.org/.

http://repo.zabbix.com/non-supported/rhel//x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
Package curl-7.29.0-46.el7.x86_64 already installed and latest version
Package wget-1.14-15.el7_4.1.x86_64 already installed and latest version
Package git-1.8.3.1-14.el7_5.x86_64 already installed and latest version
Package python-2.7.5-69.el7_5.x86_64 already installed and latest version
Package 1:make-3.82-23.el7.x86_64 already installed and latest version
Package automake-1.13.4-3.el7.noarch already installed and latest version
Package gcc-4.8.5-28.el7_5.1.x86_64 already installed and latest version
Package gcc-c++-4.8.5-28.el7_5.1.x86_64 already installed and latest version
Package kernel-plus-devel-3.10.0-862.14.4.el7.centos.plus.x86_64 already installed and latest version
Package xorg-x11-server-Xvfb-1.19.5-5.1.el7_5.0.1.x86_64 already installed and latest version
Package git-1.8.3.1-14.el7_5.x86_64 already installed and latest version
Nothing to do
done!
installing node
[LL] setting up nodejs repo....

## Installing the NodeSource Node.js 8.x LTS Carbon repo...

## Inspecting system...

+ rpm -q --whatprovides redhat-release || rpm -q --whatprovides centos-release || rpm -q --whatprovides cloudlinux-release || rpm -q --whatprovides sl-release
+ uname -m

## Confirming "el7-x86_64" is supported...

+ curl -sLf -o /dev/null 'https://rpm.nodesource.com/pub_8.x/el/7/x86_64/nodesource-release-el7-1.noarch.rpm'

## Downloading release setup RPM...

+ mktemp
+ curl -sL -o '/tmp/tmp.eu899cNxJk' 'https://rpm.nodesource.com/pub_8.x/el/7/x86_64/nodesource-release-el7-1.noarch.rpm'

## Installing release setup RPM...

+ rpm -i --nosignature --force '/tmp/tmp.eu899cNxJk'

## Cleaning up...

+ rm -f '/tmp/tmp.eu899cNxJk'

## Checking for existing installations...

+ rpm -qa 'node|npm' | grep -v nodesource

## Run `sudo yum install -y nodejs` to install Node.js 8.x LTS Carbon and npm.
## You may also need development tools to build native addons:
     sudo yum install gcc-c++ make
## To install the Yarn package manager, run:
     curl -sL https://dl.yarnpkg.com/rpm/yarn.repo | sudo tee /etc/yum.repos.d/yarn.repo
     sudo yum install yarn

done!
[LL] installing nodejs....
Failed to set locale, defaulting to C
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos.mirror.nucleus.be
 * centosplus: centos.mirror.nucleus.be
 * epel: epel.mirror.nucleus.be
 * extras: centos.mirror.nucleus.be
 * updates: centos.mirror.nucleus.be
http://repo.zabbix.com/zabbix/3.0/rhel//x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
To address this issue please refer to the below wiki article

https://wiki.centos.org/yum-errors

If above article doesn't help to resolve this issue please use https://bugs.centos.org/.

http://repo.zabbix.com/non-supported/rhel//x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
Package 2:nodejs-8.13.0-1nodesource.x86_64 already installed and latest version
Nothing to do
done!
[LL] node version - v8.13.0
[LL] yarn already installed
[LL] Running install steps
[LL] Git version: 1.8.3.1, minimum: 1.7.10
[LL] Will now try and clone the git repo for the main learninglocker software. May take some time....
running git clone
no webapp dir after git - problem
[LL] Copied example env to .env - This will need editing by hand
[LL] checking UnicodeData is present...
[LL] the path for the unicode file wasn't passed to unicode_definition_install correctly (/tmp/learninglocker/webapp/UnicodeData.txt)
done!
[LL] running yarn install....
yarn install v1.12.3
[1/4] Resolving packages...
[2/4] Fetching packages...
warning Pattern ["colour@latest"] is trying to unpack in the same destination "/usr/local/share/.cache/yarn/v4/npm-colour-0.7.1-9cb169917ec5d12c0736d3e8685746df1cadf778/node_modules/colour" as pattern ["colour@~0.7.1"]. This could result in non-deterministic behavior, skipping.
warning Pattern ["optjs@latest"] is trying to unpack in the same destination "/usr/local/share/.cache/yarn/v4/npm-optjs-3.2.2-69a6ce89c442a44403141ad2f9b370bd5bb6f4ee/node_modules/optjs" as pattern ["optjs@~3.2.2"]. This could result in non-deterministic behavior, skipping.
error https://registry.yarnpkg.com/flatmap-stream/-/flatmap-stream-0.1.1.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
done!
[LL] adding pm2....
/usr/bin/pm2 -> /usr/lib/node_modules/pm2/bin/pm2
/usr/bin/pm2-dev -> /usr/lib/node_modules/pm2/bin/pm2-dev
/usr/bin/pm2-docker -> /usr/lib/node_modules/pm2/bin/pm2-docker
/usr/bin/pm2-runtime -> /usr/lib/node_modules/pm2/bin/pm2-runtime
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.4 (node_modules/pm2/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.4: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

+ pm2@3.2.2
updated 1 package in 7.077s
done!
[LL] running yarn build-all (this can take a little while - don't worry, it's not broken)....
yarn run v1.12.3
$ babel-node api/build --release && babel-node ui/buildServer --release && babel-node ui/buildClient --release && babel-node cli/build --release && babel-node worker/build --release
/usr/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/options/option-manager.js:180
          throw new ReferenceError(messages.get("pluginUnknown", plugin, loc, i, dirname));
          ^

ReferenceError: Unknown plugin "transform-runtime" specified in "/tmp/learninglocker/webapp/.babelrc" at 0, attempted to resolve relative to "/tmp/learninglocker/webapp"
    at /usr/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/options/option-manager.js:180:17
    at Array.map (<anonymous>)
    at Function.normalisePlugins (/usr/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/options/option-manager.js:158:20)
    at OptionManager.mergeOptions (/usr/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/options/option-manager.js:234:36)
    at OptionManager.init (/usr/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/options/option-manager.js:368:12)
    at compile (/usr/lib/node_modules/babel-cli/node_modules/babel-register/lib/node.js:103:45)
    at loader (/usr/lib/node_modules/babel-cli/node_modules/babel-register/lib/node.js:144:14)
    at Object.require.extensions.(anonymous function) [as .js] (/usr/lib/node_modules/babel-cli/node_modules/babel-register/lib/node.js:154:7)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
done!
[LL] installing pm2 logrotate....
[PM2][Module] Installing NPM pm2-logrotate module
[PM2][Module] Module already installed. Updating.
[PM2] Applying action deleteProcessId on app [0](ids: 0)
[PM2] [pm2-logrotate](0) ✓
In memory process deleted
[PM2][Module] Calling [NPM] to install pm2-logrotate ...
+ pm2-logrotate@2.6.0
added 323 packages from 270 contributors and audited 2457 packages in 6.687s
found 0 vulnerabilities

[PM2][Module] Module downloaded
[PM2][WARN] Applications pm2-logrotate not running, starting...
[PM2] App [pm2-logrotate] launched (1 instances)
== pm2-logrotate ==
┌────────────────┬─────────────────────┐
│ key            │ value               │
├────────────────┼─────────────────────┤
│ max_size       │ 10M                 │
│ retain         │ 30                  │
│ compress       │ true                │
│ dateFormat     │ YYYY-MM-DD_HH-mm-ss │
│ workerInterval │ 30                  │
│ rotateInterval │ 0 0 * * *           │
│ rotateModule   │ true                │
└────────────────┴─────────────────────┘
[PM2][Module] Module successfully installed and launched
[PM2][Module] Edit configuration via: `pm2 conf`
┌──────────┬────┬─────────┬──────┬─────┬────────┬─────────┬────────┬─────┬─────┬──────┬──────────┐
│ App name │ id │ version │ mode │ pid │ status │ restart │ uptime │ cpu │ mem │ user │ watching │
└──────────┴────┴─────────┴──────┴─────┴────────┴─────────┴────────┴─────┴─────┴──────┴──────────┘
Module
┌───────────────┬────┬─────────┬───────┬────────┬─────────┬─────┬───────────┬──────┐
│ Module        │ id │ version │ pid   │ status │ restart │ cpu │ memory    │ user │
├───────────────┼────┼─────────┼───────┼────────┼─────────┼─────┼───────────┼──────┤
│ pm2-logrotate │ 0  │ 2.6.0   │ 27528 │ online │ 0       │ 0%  │ 10.3 MB   │ root │
└───────────────┴────┴─────────┴───────┴────────┴─────────┴─────┴───────────┴──────┘
 Use `pm2 show <id|name>` to get more details about an app
done!
[LL] setting up pm2 logrotate....
[PM2] Module pm2-logrotate restarted
== pm2-logrotate ==
┌────────────────┬─────────────────────┐
│ key            │ value               │
├────────────────┼─────────────────────┤
│ max_size       │ 10M                 │
│ retain         │ 30                  │
│ compress       │ true                │
│ dateFormat     │ YYYY-MM-DD_HH-mm-ss │
│ workerInterval │ 30                  │
│ rotateInterval │ 0 0 * * *           │
│ rotateModule   │ true                │
└────────────────┴─────────────────────┘
done!
[LL] running npm dedupe....
npm notice created a lockfile as package-lock.json. You should commit this file.
up to date in 1.029s
found 0 vulnerabilities

done!
[LL] Will now try and clone the git repo for XAPI. May take some time....
attempting git clone for xapi, branch: master
git clone succeeded
[LL] Copied example env to .env - This will need editing by hand
[LL] running yarn install....
yarn install v1.12.3
warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
warning "@ht2-labs/semantic-release > condition-circle@2.0.1" has incorrect peer dependency "semantic-release@^11.0.0".
[4/4] Building fresh packages...
Done in 11.28s.
done!
[LL] running yarn build....
yarn run v1.12.3
$ tsc
Done in 8.73s.
done!
[LL] creating /tmp/.tmpdist
[LL] copying modules....
can't copy directory '/tmp/learninglocker/webapp/node_modules' as it doesn't exist- exiting
asahd commented 5 years ago

Hey @jenseeckhout

Detected OS: Redhat

We don't fully support Redhat, so this is an expected issue I'm afraid - we'd highly recommend using Ubuntu 16.x instead since the script has been most-tested with this

If that's not an option, you can always follow the manual installation guide here: https://docs.learninglocker.net/guides-custom-installation/

jenseeckhout commented 5 years ago

After switching to Ubuntu 16.04.5 LTS, I'm experiencing the exact same issue. install.log is as follows:

NOTE :: Local install selected
[LL] We require a path to install to and a path to symlink to. The reason for this is that the script can be re-run in order to update
cleanly. The path we'll ask you for is a base path for the releases to be installed to so if you select the default of:
/usr/local/learninglocker/releases
then we will create a sub-directory under here for every release and symlink the latest install to the final install path (which
the nginx config points at. This is so that roll-backs can be done easier and we can perform a complete install before finally
switching the nginx config over which'll minimise downtime on upgrades
[LL] What base directory do you want to install to? (Press 'enter' for the default of /usr/local/learninglocker/releases)
attempting to use base path of: /usr/local/learninglocker/releases
[LL] What path should the release be symlinked to? (Press enter for the default of /usr/local/learninglocker/current)
attempting to use path of: /usr/local/learninglocker/current
[LL] I need a user to install the code under - what user would you like me to use? (press enter for the default of 'learninglocker')
[LL] User 'learninglocker' already exists - are you sure you want to continue? [y|n] (enter for default of 'y')
continuing using this user
[LL] MongoDB isn't installed - do you want to install it? [y|n] (press 'enter' for default of 'y')
opted to install mongo
[LL] Redis isn't installed - do you want to install it? [y|n] (press 'enter' for default of 'y')
opted to install redis
[LL] Learning Locker ideally works best with ClamAV (anti virus software) installed but it is not an absolute requirement. Do you want to install it? [y|n] (press 'enter' for the default of 'y')
user entered ''
Detected OS: Ubuntu, subver:false, arch:, vno:16.04, NodeOverride: false, PM2Override:false
[LL] Installing node version: 8.x
Reading package lists...current node version is found as v8
[LL] Node.js already installed
[LL] node version - v8.13.0
[LL] yarn already installed
[LL] Running install steps
[LL] Git version: 2.7.4, minimum: 1.7.10
[LL] Will now try and clone the git repo for the main learninglocker software. May take some time....
running git clone

Building dependency tree...
Reading state information...
Package 'cmdtest' is not installed, so not removed
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
no webapp dir after git - problem
[LL] Copied example env to .env - This will need editing by hand
[LL] checking UnicodeData is present...
[LL] the path for the unicode file wasn't passed to unicode_definition_install correctly (/tmp/learninglocker/webapp/UnicodeData.txt)
done!
[LL] running yarn install....
yarn install v1.12.3
[1/4] Resolving packages...
[2/4] Fetching packages...
warning Pattern ["colour@latest"] is trying to unpack in the same destination "/usr/local/share/.cache/yarn/v4/npm-colour-0.7.1-9cb169917ec5d12c0736d3e8685746df1cadf778/node_modules/colour" as pattern ["colour@~0.7.1"]. This could result in non-deterministic behavior, skipping.
warning Pattern ["optjs@latest"] is trying to unpack in the same destination "/usr/local/share/.cache/yarn/v4/npm-optjs-3.2.2-69a6ce89c442a44403141ad2f9b370bd5bb6f4ee/node_modules/optjs" as pattern ["optjs@~3.2.2"]. This could result in non-deterministic behavior, skipping.
error https://registry.yarnpkg.com/flatmap-stream/-/flatmap-stream-0.1.1.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
done!
[LL] adding pm2....
/usr/bin/pm2 -> /usr/lib/node_modules/pm2/bin/pm2
/usr/bin/pm2-dev -> /usr/lib/node_modules/pm2/bin/pm2-dev
/usr/bin/pm2-docker -> /usr/lib/node_modules/pm2/bin/pm2-docker
/usr/bin/pm2-runtime -> /usr/lib/node_modules/pm2/bin/pm2-runtime
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.4 (node_modules/pm2/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.4: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

+ pm2@3.2.2
updated 1 package in 6.923s
done!
[LL] running yarn build-all (this can take a little while - don't worry, it's not broken)....
yarn run v1.12.3
$ babel-node api/build --release && babel-node ui/buildServer --release && babel-node ui/buildClient --release && babel-node cli/build --release && babel-node worker/build --release
/bin/sh: 1: babel-node: not found
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
done!
[LL] installing pm2 logrotate....
[PM2][Module] Installing NPM pm2-logrotate module
[PM2][Module] Module already installed. Updating.
[PM2] Applying action deleteProcessId on app [0](ids: 0)
[PM2] [pm2-logrotate](0) ✓
In memory process deleted
[PM2][Module] Calling [NPM] to install pm2-logrotate ...
+ pm2-logrotate@2.6.0
added 323 packages from 270 contributors and audited 2457 packages in 7.796s
found 0 vulnerabilities

[PM2][Module] Module downloaded
[PM2][WARN] Applications pm2-logrotate not running, starting...
[PM2] App [pm2-logrotate] launched (1 instances)
== pm2-logrotate ==
┌────────────────┬─────────────────────┐
│ key            │ value               │
├────────────────┼─────────────────────┤
│ max_size       │ 10M                 │
│ retain         │ 30                  │
│ compress       │ true                │
│ dateFormat     │ YYYY-MM-DD_HH-mm-ss │
│ workerInterval │ 30                  │
│ rotateInterval │ 0 0 * * *           │
│ rotateModule   │ true                │
└────────────────┴─────────────────────┘
[PM2][Module] Module successfully installed and launched
[PM2][Module] Edit configuration via: `pm2 conf`
┌──────────┬────┬─────────┬──────┬─────┬────────┬─────────┬────────┬─────┬─────┬──────┬──────────┐
│ App name │ id │ version │ mode │ pid │ status │ restart │ uptime │ cpu │ mem │ user │ watching │
└──────────┴────┴─────────┴──────┴─────┴────────┴─────────┴────────┴─────┴─────┴──────┴──────────┘
Module
┌───────────────┬────┬─────────┬───────┬────────┬─────────┬─────┬───────────┬──────┐
│ Module        │ id │ version │ pid   │ status │ restart │ cpu │ memory    │ user │
├───────────────┼────┼─────────┼───────┼────────┼─────────┼─────┼───────────┼──────┤
│ pm2-logrotate │ 0  │ 2.6.0   │ 11125 │ online │ 0       │ 0%  │ 19.7 MB   │ root │
└───────────────┴────┴─────────┴───────┴────────┴─────────┴─────┴───────────┴──────┘
 Use `pm2 show <id|name>` to get more details about an app
done!
[LL] setting up pm2 logrotate....
[PM2] Module pm2-logrotate restarted
== pm2-logrotate ==
┌────────────────┬─────────────────────┐
│ key            │ value               │
├────────────────┼─────────────────────┤
│ max_size       │ 10M                 │
│ retain         │ 30                  │
│ compress       │ true                │
│ dateFormat     │ YYYY-MM-DD_HH-mm-ss │
│ workerInterval │ 30                  │
│ rotateInterval │ 0 0 * * *           │
│ rotateModule   │ true                │
└────────────────┴─────────────────────┘
done!
[LL] running npm dedupe....
npm notice created a lockfile as package-lock.json. You should commit this file.
up to date in 1.006s
found 0 vulnerabilities

done!
[LL] Will now try and clone the git repo for XAPI. May take some time....
attempting git clone for xapi, branch: master
git clone succeeded
[LL] Copied example env to .env - This will need editing by hand
[LL] running yarn install....
yarn install v1.12.3
warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
warning "@ht2-labs/semantic-release > condition-circle@2.0.1" has incorrect peer dependency "semantic-release@^11.0.0".
[4/4] Building fresh packages...
Done in 14.65s.
done!
[LL] running yarn build....
yarn run v1.12.3
$ tsc
Done in 9.07s.
done!
[LL] creating /tmp/.tmpdist
[LL] copying modules....
can't copy directory '/tmp/learninglocker/webapp/node_modules' as it doesn't exist- exiting
asahd commented 5 years ago

Fixed by https://github.com/LearningLocker/learninglocker/releases/tag/v2.6.2, @jenseeckhout has confirmed