ContentMine / quickscrape

A scraping command line tool for the modern web
MIT License
259 stars 42 forks source link

Lubuntu 15.04 install failed (?) #41

Closed rossmounce closed 9 years ago

rossmounce commented 9 years ago
$ sudo npm install --global quickscrape
-
> phantomjs@1.9.16 install /usr/lib/node_modules/quickscrape/node_modules/thresher/node_modules/phantomjs
> node install.js

PhantomJS detected, but wrong version 1.9.0 @ /usr/bin/phantomjs.
Downloading https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-1.9.8-linux-x86_64.tar.bz2
Saving to /usr/lib/node_modules/quickscrape/node_modules/thresher/node_modules/phantomjs/phantomjs/phantomjs-1.9.8-linux-x86_64.tar.bz2
Receiving...
  [========================================] 100% 0.0s
Received 12854K total.
Extracting tar contents (via spawned process)
Removing /usr/lib/node_modules/quickscrape/node_modules/thresher/node_modules/phantomjs/lib/phantom
Copying extracted folder /usr/lib/node_modules/quickscrape/node_modules/thresher/node_modules/phantomjs/phantomjs/phantomjs-1.9.8-linux-x86_64.tar.bz2-extract-1430748442293/phantomjs-1.9.8-linux-x86_64 -> /usr/lib/node_modules/quickscrape/node_modules/thresher/node_modules/phantomjs/lib/phantom
Writing location.js file
Done. Phantomjs binary available at /usr/lib/node_modules/quickscrape/node_modules/thresher/node_modules/phantomjs/lib/phantom/bin/phantomjs
/usr/bin/quickscrape -> /usr/lib/node_modules/quickscrape/bin/quickscrape.js
quickscrape@0.4.1 /usr/lib/node_modules/quickscrape
├── which@1.0.9
├── commander@2.7.1 (graceful-readlink@1.0.1)
├── chalk@1.0.0 (escape-string-regexp@1.0.3, ansi-styles@2.0.1, supports-color@1.3.1, strip-ansi@2.0.1, has-ansi@1.0.3)
├── winston@1.0.0 (cycle@1.0.3, stack-trace@0.0.9, eyes@0.1.8, isstream@0.1.2, pkginfo@0.3.0, async@0.9.0, colors@1.0.3)
├── moment@2.10.2
└── thresher@0.1.3 (underscore-deep-extend@0.0.5, set@1.1.1, eventemitter2@0.4.14, request-progress@0.3.1, shelljs@0.4.0, tough-cookie@0.12.1, spooky@0.2.5, request@2.54.0, casperjs@1.1.0-beta3, lodash@3.8.0, jsdom-little@0.10.6, download@4.1.2, phantomjs@1.9.16)

$ quickscrape # tab auto-complete works
$ quickscrape --help
$ [no output/response]
$ phantomjs --version
1.9.0
$ nodejs --version
v0.10.33

Also tried installing after installing NVM

curl https://raw.githubusercontent.com/creationix/nvm/v0.24.1/install.sh | bash

but no dice with that either

rossmounce commented 9 years ago

Reminds me of an earlier issue: https://github.com/ContentMine/quickscrape/issues/3

blahah commented 9 years ago

The current install instructions do not include a sudo, because that breaks node in general.

Once npm is installed, you don't need a sudo. Can you try running the instructions exactly as in the README?

rossmounce commented 9 years ago

After installing npm the easy Lubuntu way (sudo apt-get install npm)

$ npm --version
1.4.21
$ npm install --global quickscrape
npm ERR! Error: EACCES, mkdir '/usr/local/lib/node_modules'
npm ERR!  { [Error: EACCES, mkdir '/usr/local/lib/node_modules']
npm ERR!   errno: 3,
npm ERR!   code: 'EACCES',
npm ERR!   path: '/usr/local/lib/node_modules',
npm ERR!   fstream_type: 'Directory',
npm ERR!   fstream_path: '/usr/local/lib/node_modules/quickscrape',
npm ERR!   fstream_class: 'DirWriter',
npm ERR!   fstream_stack: 
npm ERR!    [ '/usr/lib/nodejs/fstream/lib/writer.js:171:23',
npm ERR!      '/usr/lib/nodejs/mkdirp/index.js:46:53',
npm ERR!      'Object.oncomplete (fs.js:107:15)' ] }
npm ERR! 
npm ERR! Please try running this command again as root/Administrator.

npm ERR! System Linux 3.19.0-15-generic
npm ERR! command "/usr/bin/nodejs" "/usr/bin/npm" "install" "--global" "quickscrape"
npm ERR! cwd /home/ross/Dropbox/2015/journals/torrey/pdfs05to15
npm ERR! node -v v0.10.25
npm ERR! npm -v 1.4.21
npm ERR! path /usr/local/lib/node_modules
npm ERR! fstream_path /usr/local/lib/node_modules/quickscrape
npm ERR! fstream_type Directory
npm ERR! fstream_class DirWriter
npm ERR! code EACCES
npm ERR! errno 3
npm ERR! stack Error: EACCES, mkdir '/usr/local/lib/node_modules'
npm ERR! fstream_stack /usr/lib/nodejs/fstream/lib/writer.js:171:23
npm ERR! fstream_stack /usr/lib/nodejs/mkdirp/index.js:46:53
npm ERR! fstream_stack Object.oncomplete (fs.js:107:15)
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /home/ross/Dropbox/2015/journals/torrey/pdfs05to15/npm-debug.log
npm ERR! not ok code 0
blahah commented 9 years ago

Odd that Lubuntu would have nvm in their package manager, because that leads to users installing it using sudo, which breaks it.

I think if you install npm with sudo I think you have to install node global packages with sudo too, and then you probably have to run those packages with sudo.

How about:

sudo npm install --global quickscrape
sudo quickscrape --help
rossmounce commented 9 years ago

first step fails.

npm-debug log is too big for paste bin. It's 1.4Mb !!! The npm-debug log is here: https://www.dropbox.com/s/bs2apm48iorgt53/npm-debug.log?dl=0

Would be good to have instructions for installing npm on *buntu me thinks. Is there another way I should have installed it? Willing to try that if it would be useful?

blahah commented 9 years ago

Yes, there are instructions for installing npm on any unix platform in the README: https://github.com/ContentMine/quickscrape#installation.

I think using your existing setup you might need to do:

sudo npm install --global quickscrape --unsafe-perm
quickscrape --help

This is because the npm is not meant to be managed under super user priveleges, so it shouldn't be installed via a normal package manager. Should I make the README more clear?

rossmounce commented 9 years ago

Yes, please make the README more clear. If I'm not mistaken, the first instruction under the Installation heading is

npm install --global quickscrape

yet *buntu doesn't come with npm installed, so instructions for installing npm are required.

I will try the unsafe-perm flag & get back to you ASAP

blahah commented 9 years ago

If you read the lines after that command, it tells you how to install nvm and node which will automatically install npm. My thinking was that people will read that line and if they understand it, they can just run it. If they don't understand they will keep reading and the next paragraph or so tells them how to install nvm and node.

But perhaps when people see the command and they know they don't have it installed, they go looking elsewhere for how to install it. So maybe I should reorder it to put the nvm/node instructions first?

Also, maybe I should say explicitly that npm will be installed along with node.

rossmounce commented 9 years ago

unsafe-perms didn't work either: https://www.dropbox.com/s/y7xhgxwwh9wku9o/npm-debug.log?dl=0

rossmounce commented 9 years ago

I generally prefer to have install instructions in completely chronological order. I guess I failed to read downwards to see the pre-install instructions. It's also not clear when quickly scanning that 'NVM' or 'Node' have anything to do with 'npm'

Please do put the nvm / node instructions first, perhaps even in a pre-requisites / dependencies section. Also make clear that nvm / node install allows you to run 'npm'

blahah commented 9 years ago

OK, full instructions to get rid of this setup and get a clean one:

sudo apt-get -y remove npm nodejs
curl https://raw.githubusercontent.com/creationix/nvm/v0.24.1/install.sh | bash

then wait for the NVM install to complete, then run:

source ~/.nvm/nvm.sh
nvm install 0.10
nvm alias default 0.10
nvm use default
npm install --global quickscrape

Any luck?

blahah commented 9 years ago

OK, will do.

rossmounce commented 9 years ago

aha! success!

$ quickscrape --version
0.4.1

just being pedantic now, but FYI curl isn't installed by default either

blahah commented 9 years ago

Great!

Some systems come with curl by default (e.g. OSX), some with wget (e.g. Ubuntu), so in the README I gave both. Any more feedback on the install instructions is welcome :)

blahah commented 9 years ago

... but please start a new issue - going to close this one as your install is fixed.