Closed blahah closed 10 years ago
@petermr seems like a problem with the nodejs installation. Can you post the output of the following:
node --version
nodejs
ls -alh /Users/pm286/.node-gyp/
node --version v0.10.28 okapi:quickscrape pm286$ nodejs -bash: nodejs: command not found okapi:quickscrape pm286$ ls -alh /Users/pm286/.node-gyp/ ls: /Users/pm286/.node-gyp/: No such file or directory okapi:quickscrape pm286$
On Sat, May 31, 2014 at 1:57 PM, Richard Smith-Unna < notifications@github.com> wrote:
@petermr https://github.com/petermr seems like a problem with the nodejs installation. Can you post the output of the following:
- node --version
- nodejs
- ls -alh /Users/pm286/.node-gyp/
— Reply to this email directly or view it on GitHub https://github.com/ContentMine/quickscrape/issues/1#issuecomment-44746384 .
Peter Murray-Rust Reader in Molecular Informatics Unilever Centre, Dep. Of Chemistry University of Cambridge CB2 1EW, UK +44-1223-763069
OK, the problem is caused by a dependency we're actually not using anymore. I'll remove them from the next version and ping you when it gets pushed (in the next couple of hours).
Brill, (spent a day writing the same library as I did 5 years ago! Happy to test your stuff)
On Sat, May 31, 2014 at 2:56 PM, Richard Smith-Unna < notifications@github.com> wrote:
OK, the problem is caused by a dependency we're actually not using anymore. I'll remove them from the next version and ping you when it gets pushed (in the next couple of hours).
— Reply to this email directly or view it on GitHub https://github.com/ContentMine/quickscrape/issues/1#issuecomment-44747607 .
Peter Murray-Rust Reader in Molecular Informatics Unilever Centre, Dep. Of Chemistry University of Cambridge CB2 1EW, UK +44-1223-763069
You spent a day writing or I did? It's a shame how often things don't get reused.
In this case I've leant very heavily on existing Node.js packages, but in the stuff I've written from scratch there's necessarily some duplication from existing scrapers. The problem is that it's harder to use what they've created than to write it from scratch. And if I write it myself, I can document and test it properly (the fact that others don't do this is part of why I rewrote).
Sorry - nothing to do with you :-) I spent a day on train writing my OCR stuff and creating a library which I then found I had already written! No harm, just I could have gone further...
On Sat, May 31, 2014 at 3:35 PM, Richard Smith-Unna < notifications@github.com> wrote:
You spent a day writing or I did? It's a shame how often things don't get reused.
In this case I've leant very heavily on existing Node.js packages, but in the stuff I've written from scratch there's necessarily some duplication from existing scrapers. The problem is that it's harder to use what they've created than to write it from scratch. And if I write it myself, I can document and test it properly (the fact that others don't do this is part of why I rewrote).
— Reply to this email directly or view it on GitHub https://github.com/ContentMine/quickscrape/issues/1#issuecomment-44748477 .
Peter Murray-Rust Reader in Molecular Informatics Unilever Centre, Dep. Of Chemistry University of Cambridge CB2 1EW, UK +44-1223-763069
Haha yeah I do that frequently too. Good for exercising the creative brain, if not the memory!
OK, I've pushed version 0.1.4. Please try again (sudo npm install --global quickscrape
), and see the README on the project homepage for updated usage instructions.
Message received - am hacking it. (Wifi very slow here)
Thanks
On Sat, May 31, 2014 at 9:29 PM, Richard Smith-Unna < notifications@github.com> wrote:
OK, I've pushed version 0.1.4. Please try again (sudo npm install --global quickscrape), and see the README on the project homepage for updated usage instructions.
— Reply to this email directly or view it on GitHub https://github.com/ContentMine/quickscrape/issues/1#issuecomment-44757341 .
Peter Murray-Rust Reader in Molecular Informatics Unilever Centre, Dep. Of Chemistry University of Cambridge CB2 1EW, UK +44-1223-763069
Afraid it failed again. Will post to issues hopefully
On Sat, May 31, 2014 at 9:55 PM, Peter Murray-Rust pm286@cam.ac.uk wrote:
Message received - am hacking it. (Wifi very slow here)
Thanks
On Sat, May 31, 2014 at 9:29 PM, Richard Smith-Unna < notifications@github.com> wrote:
OK, I've pushed version 0.1.4. Please try again (sudo npm install --global quickscrape), and see the README on the project homepage for updated usage instructions.
— Reply to this email directly or view it on GitHub https://github.com/ContentMine/quickscrape/issues/1#issuecomment-44757341 .
Peter Murray-Rust Reader in Molecular Informatics Unilever Centre, Dep. Of Chemistry University of Cambridge CB2 1EW, UK +44-1223-763069
Peter Murray-Rust Reader in Molecular Informatics Unilever Centre, Dep. Of Chemistry University of Cambridge CB2 1EW, UK +44-1223-763069
okapi:quickscrape pm286$ sudo npm install --global quickscrape Password: npm http GET https://registry.npmjs.org/quickscrape npm http 200 https://registry.npmjs.org/quickscrape npm http GET https://registry.npmjs.org/quickscrape/-/quickscrape-0.1.4.tgz npm http 200 https://registry.npmjs.org/quickscrape/-/quickscrape-0.1.4.tgz npm http GET https://registry.npmjs.org/commander npm http GET https://registry.npmjs.org/spooky npm http GET https://registry.npmjs.org/download npm http GET https://registry.npmjs.org/xpath npm http GET https://registry.npmjs.org/jsdom npm http GET https://registry.npmjs.org/winston npm http GET https://registry.npmjs.org/which npm http 304 https://registry.npmjs.org/download npm http 304 https://registry.npmjs.org/jsdom npm http 304 https://registry.npmjs.org/spooky npm http 304 https://registry.npmjs.org/xpath npm http 304 https://registry.npmjs.org/commander npm http 304 https://registry.npmjs.org/which npm http 304 https://registry.npmjs.org/winston npm http GET https://registry.npmjs.org/decompress npm http GET https://registry.npmjs.org/each-async npm http GET https://registry.npmjs.org/get-stdin npm http GET https://registry.npmjs.org/get-urls npm http GET https://registry.npmjs.org/mkdirp npm http GET https://registry.npmjs.org/nopt npm http GET https://registry.npmjs.org/request npm http GET https://registry.npmjs.org/through2 npm http GET https://registry.npmjs.org/htmlparser2 npm http GET https://registry.npmjs.org/request npm http GET https://registry.npmjs.org/nwmatcher npm http GET https://registry.npmjs.org/xmlhttprequest npm http GET https://registry.npmjs.org/cssom npm http GET https://registry.npmjs.org/cssstyle npm http GET https://registry.npmjs.org/contextify npm http GET https://registry.npmjs.org/underscore npm http GET https://registry.npmjs.org/async npm http GET https://registry.npmjs.org/tiny-jsonrpc npm http GET https://registry.npmjs.org/carrier npm http GET https://registry.npmjs.org/duplexer npm http GET https://registry.npmjs.org/readable-stream npm http 304 https://registry.npmjs.org/decompress npm http GET https://registry.npmjs.org/async npm http GET https://registry.npmjs.org/colors npm http GET https://registry.npmjs.org/cycle npm http GET https://registry.npmjs.org/eyes npm http GET https://registry.npmjs.org/pkginfo npm http GET https://registry.npmjs.org/request npm http GET https://registry.npmjs.org/stack-trace npm http 304 https://registry.npmjs.org/nopt npm http 304 https://registry.npmjs.org/get-urls npm http 304 https://registry.npmjs.org/request npm http 304 https://registry.npmjs.org/each-async npm http 304 https://registry.npmjs.org/mkdirp npm http 304 https://registry.npmjs.org/request npm http 304 https://registry.npmjs.org/through2 npm http 304 https://registry.npmjs.org/get-stdin npm http 304 https://registry.npmjs.org/htmlparser2 npm http 304 https://registry.npmjs.org/nwmatcher npm http 304 https://registry.npmjs.org/xmlhttprequest npm http 304 https://registry.npmjs.org/cssom npm http 304 https://registry.npmjs.org/underscore npm http 304 https://registry.npmjs.org/async npm http 304 https://registry.npmjs.org/contextify npm http GET https://registry.npmjs.org/adm-zip npm http GET https://registry.npmjs.org/extname npm http GET https://registry.npmjs.org/rimraf npm http GET https://registry.npmjs.org/stream-combiner npm http GET https://registry.npmjs.org/tar npm http GET https://registry.npmjs.org/tempfile npm http GET https://registry.npmjs.org/map-key npm http 304 https://registry.npmjs.org/cssstyle npm http GET https://registry.npmjs.org/readable-stream npm http GET https://registry.npmjs.org/xtend npm http 304 https://registry.npmjs.org/duplexer npm http GET https://registry.npmjs.org/abbrev npm http 304 https://registry.npmjs.org/async npm http 304 https://registry.npmjs.org/readable-stream npm http 304 https://registry.npmjs.org/carrier npm http 304 https://registry.npmjs.org/tiny-jsonrpc npm http 304 https://registry.npmjs.org/request npm http 304 https://registry.npmjs.org/eyes npm http 304 https://registry.npmjs.org/colors npm http 304 https://registry.npmjs.org/pkginfo npm http 304 https://registry.npmjs.org/cycle npm http GET https://registry.npmjs.org/qs npm http GET https://registry.npmjs.org/json-stringify-safe npm http GET https://registry.npmjs.org/forever-agent npm http GET https://registry.npmjs.org/mime npm http GET https://registry.npmjs.org/node-uuid npm http GET https://registry.npmjs.org/tough-cookie npm http GET https://registry.npmjs.org/form-data npm http 304 https://registry.npmjs.org/extname npm http 304 https://registry.npmjs.org/rimraf npm http GET https://registry.npmjs.org/tunnel-agent npm http GET https://registry.npmjs.org/http-signature npm http GET https://registry.npmjs.org/oauth-sign npm http GET https://registry.npmjs.org/hawk npm http GET https://registry.npmjs.org/aws-sign2 npm http 304 https://registry.npmjs.org/adm-zip npm http 304 https://registry.npmjs.org/stream-combiner npm http 304 https://registry.npmjs.org/readable-stream npm http 304 https://registry.npmjs.org/tempfile npm http 304 https://registry.npmjs.org/tar npm http 304 https://registry.npmjs.org/stack-trace npm http 304 https://registry.npmjs.org/map-key npm http GET https://registry.npmjs.org/bindings npm http GET https://registry.npmjs.org/nan npm http 304 https://registry.npmjs.org/xtend npm http 304 https://registry.npmjs.org/abbrev npm http 304 https://registry.npmjs.org/json-stringify-safe npm http 304 https://registry.npmjs.org/forever-agent npm http 304 https://registry.npmjs.org/mime npm http 304 https://registry.npmjs.org/node-uuid npm http 304 https://registry.npmjs.org/tough-cookie npm http 304 https://registry.npmjs.org/qs npm http 304 https://registry.npmjs.org/tunnel-agent npm http 304 https://registry.npmjs.org/http-signature npm http 304 https://registry.npmjs.org/oauth-sign npm http 304 https://registry.npmjs.org/form-data npm http 304 https://registry.npmjs.org/hawk npm http GET https://registry.npmjs.org/uuid npm http GET https://registry.npmjs.org/lodash npm http GET https://registry.npmjs.org/underscore.string npm http GET https://registry.npmjs.org/underscore.string npm http GET https://registry.npmjs.org/ext-list npm http 304 https://registry.npmjs.org/aws-sign2 npm http 304 https://registry.npmjs.org/nan npm http 304 https://registry.npmjs.org/bindings npm http GET https://registry.npmjs.org/inherits npm http GET https://registry.npmjs.org/core-util-is npm http GET https://registry.npmjs.org/isarray npm http GET https://registry.npmjs.org/string_decoder npm http 304 https://registry.npmjs.org/underscore.string npm http 304 https://registry.npmjs.org/lodash npm http 304 https://registry.npmjs.org/ext-list npm http 304 https://registry.npmjs.org/inherits npm http 304 https://registry.npmjs.org/isarray npm http 304 https://registry.npmjs.org/uuid npm http GET https://registry.npmjs.org/object-keys npm http 304 https://registry.npmjs.org/core-util-is npm http 304 https://registry.npmjs.org/string_decoder npm http 304 https://registry.npmjs.org/underscore.string npm http 304 https://registry.npmjs.org/object-keys npm http GET https://registry.npmjs.org/combined-stream npm http GET https://registry.npmjs.org/assert-plus npm http GET https://registry.npmjs.org/asn1 npm http GET https://registry.npmjs.org/ctype
contextify@0.1.8 install /usr/local/lib/node_modules/quickscrape/node_modules/jsdom/node_modules/contextify node-gyp rebuild
gyp WARN EACCES user "root" does not have permission to access the dev dir "/Users/pm286/.node-gyp/0.10.28" gyp WARN EACCES attempting to reinstall using temporary dev dir "/usr/local/lib/node_modules/quickscrape/node_modules/jsdom/node_modules/contextify/.node-gyp" gyp http GET http://nodejs.org/dist/v0.10.28/node-v0.10.28.tar.gz gyp http 200 http://nodejs.org/dist/v0.10.28/node-v0.10.28.tar.gz
gyp ERR! Completion callback never invoked!
gyp ERR! System Darwin 13.2.0
gyp ERR! command "node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /usr/local/lib/node_modules/quickscrape/node_modules/jsdom/node_modules/contextify
gyp ERR! node -v v0.10.28
gyp ERR! node-gyp -v v0.13.0
gyp ERR! This is a bug in node-gyp
.
gyp ERR! Try to update node-gyp and file an Issue if it does not help:
gyp ERR! https://github.com/TooTallNate/node-gyp/issues
npm http 304 https://registry.npmjs.org/asn1
npm http 304 https://registry.npmjs.org/assert-plus
npm http 304 https://registry.npmjs.org/ctype
npm http GET https://registry.npmjs.org/fstream
npm http GET https://registry.npmjs.org/block-stream
npm http GET https://registry.npmjs.org/domhandler
npm http GET https://registry.npmjs.org/domutils
npm http GET https://registry.npmjs.org/domelementtype
npm http GET https://registry.npmjs.org/entities
npm http 304 https://registry.npmjs.org/combined-stream
npm http GET https://registry.npmjs.org/punycode
npm http GET https://registry.npmjs.org/cookie-jar
npm http GET https://registry.npmjs.org/aws-sign
npm http 304 https://registry.npmjs.org/block-stream
npm http 304 https://registry.npmjs.org/domelementtype
npm http GET https://registry.npmjs.org/hoek
npm http GET https://registry.npmjs.org/boom
npm http GET https://registry.npmjs.org/cryptiles
npm http GET https://registry.npmjs.org/sntp
npm http 304 https://registry.npmjs.org/fstream
npm http 304 https://registry.npmjs.org/domhandler
npm http 304 https://registry.npmjs.org/domutils
npm http 304 https://registry.npmjs.org/punycode
npm http 304 https://registry.npmjs.org/aws-sign
npm http 304 https://registry.npmjs.org/cookie-jar
npm http 304 https://registry.npmjs.org/hoek
npm http GET https://registry.npmjs.org/delayed-stream
npm http 304 https://registry.npmjs.org/boom
npm http 304 https://registry.npmjs.org/cryptiles
npm http 304 https://registry.npmjs.org/sntp
npm WARN engine hawk@0.10.2: wanted: {"node":"0.8.x"} (current: {"node":"v0.10.28","npm":"1.4.9"})
npm http 304 https://registry.npmjs.org/entities
npm http 304 https://registry.npmjs.org/delayed-stream
npm http GET https://registry.npmjs.org/graceful-fs
npm http 304 https://registry.npmjs.org/graceful-fs
npm WARN engine hoek@0.7.6: wanted: {"node":"0.8.x"} (current: {"node":"v0.10.28","npm":"1.4.9"})
npm WARN engine boom@0.3.8: wanted: {"node":"0.8.x"} (current: {"node":"v0.10.28","npm":"1.4.9"})
npm WARN engine cryptiles@0.1.3: wanted: {"node":"0.8.x"} (current: {"node":"v0.10.28","npm":"1.4.9"})
npm WARN engine sntp@0.1.4: wanted: {"node":"0.8.x"} (current: {"node":"v0.10.28","npm":"1.4.9"})
npm ERR! contextify@0.1.8 install: node-gyp rebuild
npm ERR! Exit status 6
npm ERR!
npm ERR! Failed at the contextify@0.1.8 install script.
npm ERR! This is most likely a problem with the contextify package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node-gyp rebuild
npm ERR! You can get their info via:
npm ERR! npm owner ls contextify
npm ERR! There is likely additional logging output above.
npm ERR! System Darwin 13.2.0 npm ERR! command "node" "/usr/local/bin/npm" "install" "--global" "quickscrape" npm ERR! cwd /Users/pm286/workspace/quickscrape npm ERR! node -v v0.10.28 npm ERR! npm -v 1.4.9 npm ERR! code ELIFECYCLE npm ERR! npm ERR! Additional logging details can be found in: npm ERR! /Users/pm286/workspace/quickscrape/npm-debug.log npm ERR! not ok code 0
should I create a ./node-gyp directory?
Not sure what's going on there yet, can you post contents of /Users/pm286/workspace/quickscrape/npm-debug.log
?
also can you try sudo npm install --global --unsafe-perm quickscrape
?
There appears to be a known bug that causes something similar to this: https://github.com/TooTallNate/node-gyp/issues/115#issuecomment-28490552
final thing to try is:
env SUDO_UID= SUDO_USER= npm i
sudo npm install --global quckscrape
Many Thanks Richard,
Will do those things now.
Questions:
P.
On Sun, Jun 1, 2014 at 12:54 AM, Richard Smith-Unna < notifications@github.com> wrote:
Not sure what's going on there yet, can you post contents of /Users/pm286/workspace/quickscrape/npm-debug.log?
— Reply to this email directly or view it on GitHub https://github.com/ContentMine/quickscrape/issues/1#issuecomment-44762511 .
Peter Murray-Rust Reader in Molecular Informatics Unilever Centre, Dep. Of Chemistry University of Cambridge CB2 1EW, UK +44-1223-763069
Richard, It's 2 MB so I have mailed it - hope that OK.
Suggest we get Ross and Mark to try it.
On Sun, Jun 1, 2014 at 8:45 AM, Peter Murray-Rust pm286@cam.ac.uk wrote:
Many Thanks Richard,
Will do those things now.
Questions:
- is this a Mac OSX issue or generally? i.e. can we kludge round by finding "pure" unix people in each group. (I am assuming it worksForYou; maybe get MarkM and Ross to beta test ...
- shall I reinstall from scratch? If so suggest you give the complete list of commands , possibly including an initial "clean"
P.
On Sun, Jun 1, 2014 at 12:54 AM, Richard Smith-Unna < notifications@github.com> wrote:
Not sure what's going on there yet, can you post contents of /Users/pm286/workspace/quickscrape/npm-debug.log?
— Reply to this email directly or view it on GitHub https://github.com/ContentMine/quickscrape/issues/1#issuecomment-44762511 .
Peter Murray-Rust Reader in Molecular Informatics Unilever Centre, Dep. Of Chemistry University of Cambridge CB2 1EW, UK +44-1223-763069
Peter Murray-Rust Reader in Molecular Informatics Unilever Centre, Dep. Of Chemistry University of Cambridge CB2 1EW, UK +44-1223-763069
PROGRESS!
Seems to not fail. tail of console:
/usr/local/bin/quickscrape -> /usr/local/lib/node_modules/quickscrape/bin/quickscrape.js
quickscrape@0.1.4 /usr/local/lib/node_modules/quickscrape
├── which@1.0.5
├── commander@2.2.0
├── xpath@0.0.6
├── spooky@0.2.4 (duplexer@0.0.4, carrier@0.1.14, async@0.1.22, underscore@1.3.3, tiny-jsonrpc@0.2.1, readable-stream@1.0.27-1)
├── winston@0.7.3 (cycle@1.0.3, stack-trace@0.0.9, eyes@0.1.8, colors@0.6.2, async@0.2.10, pkginfo@0.3.0, request@2.16.6)
├── download@0.1.17 (get-stdin@0.1.0, each-async@0.1.3, get-urls@0.1.2, mkdirp@0.3.5, nopt@2.2.1, through2@0.4.2, decompress@0.2.3, request@2.36.0)
└── jsdom@0.10.6 (xmlhttprequest@1.6.0, cssom@0.3.0, nwmatcher@1.3.3, htmlparser2@3.7.2, cssstyle@0.2.14, request@2.36.0, contextify@0.1.8)
okapi:quickscrape pm286$
Will try to run it and get back to you..
VERY much appreciated
On Sun, Jun 1, 2014 at 8:56 AM, Peter Murray-Rust pm286@cam.ac.uk wrote:
Richard, It's 2 MB so I have mailed it - hope that OK.
Suggest we get Ross and Mark to try it.
On Sun, Jun 1, 2014 at 8:45 AM, Peter Murray-Rust pm286@cam.ac.uk wrote:
Many Thanks Richard,
Will do those things now.
Questions:
- is this a Mac OSX issue or generally? i.e. can we kludge round by finding "pure" unix people in each group. (I am assuming it worksForYou; maybe get MarkM and Ross to beta test ...
- shall I reinstall from scratch? If so suggest you give the complete list of commands , possibly including an initial "clean"
P.
On Sun, Jun 1, 2014 at 12:54 AM, Richard Smith-Unna < notifications@github.com> wrote:
Not sure what's going on there yet, can you post contents of /Users/pm286/workspace/quickscrape/npm-debug.log?
— Reply to this email directly or view it on GitHub https://github.com/ContentMine/quickscrape/issues/1#issuecomment-44762511 .
Peter Murray-Rust Reader in Molecular Informatics Unilever Centre, Dep. Of Chemistry University of Cambridge CB2 1EW, UK +44-1223-763069
Peter Murray-Rust Reader in Molecular Informatics Unilever Centre, Dep. Of Chemistry University of Cambridge CB2 1EW, UK +44-1223-763069
Peter Murray-Rust Reader in Molecular Informatics Unilever Centre, Dep. Of Chemistry University of Cambridge CB2 1EW, UK +44-1223-763069
Looks like that has worked. Not clear to me yet whether it's OSX specific.
It's not OSX-specific, but the --unsafe-perms
flag fixes it for now. We'll have to update as the the package manager gets fixed, but I'll close it in the meantime.
from @petermr, moved from http://pads.cottagelabs.com/p/contentminescraping