search

robobario / gameswap

0 stars 1 forks source link

scrape game data from BGG #2

Open robobario opened 8 years ago

robobario commented 8 years ago

make the worker process periodically scrape new game info from BGG and save it into the db.

We want.

Game name, game rank if available, id

example: https://www.boardgamegeek.com/xmlapi2/thing?type=boardgame&id=1,2,3&stats=1

robobario commented 8 years ago

we should try to discover the max batch size and be kind to their API, also our worker has to sleep a reasonable amount to be free on heroku