pstoica / pow-viz

MIT License
2 stars 1 forks source link

Make a task that fills Mongo collection with price history #6

Closed pstoica closed 10 years ago

pstoica commented 10 years ago

Maybe define a Grunt task that loads the CSV on zmjones' site and converts those into Mongo records. There should be a last_updated variable to know when things have already been added.

Also, I just realized there may be multiple entries per state per day, so do we need to aggregate results?

pstoica commented 10 years ago

Don't use grunt, just make a separate file to run in node or whatever.

pstoica commented 10 years ago

Here's a csv parser: https://github.com/wdavidw/node-csv

vis-123 commented 10 years ago

ok, so to run the task, its node csv_importer.js

and, the format for the documents is something like this: { lon: csv[0], lat: csv[9], city: csv[1], state: csv[2], price: csv[3], amount: csv[4], quality: csv[5], date: new Date(csv[6]), ppg: csv[7], state_name: csv[8] }