davidchambers / tutor

JavaScript interface for the Gatherer card database
https://gatherer.wizards.com/
Do What The F*ck You Want To Public License
149 stars 18 forks source link

Add node.js and node_module dependencies to the README #61

Closed ire-and-curses closed 11 years ago

ire-and-curses commented 11 years ago

I'm no Javascript programmer, but your command line API (with json output) is just what I was looking for to set up the DB of a MTG inventory project I'm coding in Python. It took me a while to get a working build though, because of the node.js module dependencies.

The code didn't seem to be finding the bundled node executable. I've never used node.js before, but I grabbed it from the project homepage and installed globally to /usr/local/bin, which was fine.

I discovered each dependency by running the Makefile and waiting for module missing messages. Initially I installed the node.js modules globally. However, they were not picked up by the Makefile, which expects the modules in a local dir. I tried changing the bin dir to /usr/local/bin, which partially worked, but the Makefile couldn't find the nock install (which was only in /usr/local/lib/node_modules I believe). Wasn't sure how to hack that, so I backtracked and reinstalled to the node_modules directory indicated in the Makefile.

In the end, from a pristine git clone, I had to do these additional steps in order to get make/make test to run without crashing:

mkdir node_modules
npm install mocha
npm install coffee-script
npm install nock
npm install request
npm install entities
npm install cheerio
npm install commander

If something to this effect could be added to the README, it would save time for future node.js noobs like myself.

Side note 1: I had three failing unit tests, but the CLI tests all passed, and it seems to be working for the queries I am making. Thanks very much for this project! I can report them separately if that is useful.

Side note 2: It would be awesome if the rest of the Javascript API was accessible via the CLI, since this would open it up to calls from other languages. For example, I would make use of the set name listing were it available (I tried blindly hacking it in, following the pattern of the other options (minus the argument), but was unsuccessful).

davidchambers commented 11 years ago

It shouldn't be necessary to install dependencies manually. make setup runs npm install, which should install the correct version of each dependency (as specified in package.json). The following steps should be sufficient to install Tutor from source:

git clone git@github.com:davidchambers/tutor.git
cd tutor
make setup

One more step is required in order to run the tests:

make fixtures

Side note 1: I had three failing unit tests, but the CLI tests all passed, and it seems to be working for the queries I am making. Thanks very much for this project! I can report them separately if that is useful.

Scraping is a fickle business. The tests all pass in my local clone with fixtures generated some time ago, but three tests fail in my /tmp/tutor clone with freshly generated fixtures. I'll diff the fixtures to see how the markup has changed, and (hopefully) push out a fix.

I'm pleased you stumbled upon this project. Ideally Wizards of the Coast would provide a public API, but in lieu of that Tutor prevents every MTG app developer from having to write his own scraper.

Side note 2: It would be awesome if the rest of the Javascript API was accessible via the CLI, since this would open it up to calls from other languages. For example, I would make use of the set name listing were it available (I tried blindly hacking it in, following the pattern of the other options (minus the argument), but was unsuccessful).

Great point. The CLI should expose all the functionality of the JavaScript API. Open an issue for this if you feel so inclined.

ire-and-curses commented 11 years ago

Ah. I missed the make setup goal in the Makefile. Perhaps you could add an INSTALL or similar file with those steps listed (or I can send it to you as a patch)?

Yes, the lack of a public Gatherer API is a real pain, and there is a clear need. I think a lot of developers go through the API at TCGplayer, but that's not open source and not freely available without special access either.

I'll open a separate issue for the CLI, as you suggest.

davidchambers commented 11 years ago

I'll diff the fixtures to see how the markup has changed, and (hopefully) push out a fix.

7c540d361c688900da85b42e419c5322689a8bce fixes the issue that was causing tests to fail. If you pull from master and run make fixtures, all the tests should then pass.

I'll close this issue, but feel free to open a pull request adding an INSTALL file if you feel it would improve discoverability.