search

edgecase / bird-wave

An api for eBird data
0 stars 0 forks source link

Modify schema. Add route for species counts. #4

Closed Bestra closed 10 years ago

Bestra commented 10 years ago

Database related changes

tennety commented 10 years ago

Somehow the /species route is broken. It seems like the connection is returning nil.

jxa commented 10 years ago

@tennety is bird-man.service/datomic-connection nil? you might need to call (go). Make sure you call it with the argument that means "don't reload the database"! (go false) maybe ?

Bestra commented 10 years ago

I should split out the database seed from the server/db initialization completely to avoid having to modify the code based on the use case.

tennety commented 10 years ago

@jxa Yes, datomic-connection was nil. I used (go). I can show you guys tomorrow sometime.

Update: Running with (go false) worked. It definitely didn't try to reload the schema earlier though, there was probably an exception I didn't see! @Bestra we kinda did separate it out (see my commit above). The init method now takes a boolean on whether or not to seed the db.

jxa commented 10 years ago

There is now a separate taxon model and it should be fairly fast to retrieve the index of birds now. Now I just want to test the import on the huge file to make sure it can handle duplicate taxons in the same "chunk".

jxa commented 10 years ago

P.S. Can we just do this as our job? I love this project!

tennety commented 10 years ago

:+1:

jxa commented 10 years ago

This is ready for review and merge to master. I'm backing up my successful import to an S3 bucket and I emailed you each credentials for accessing it. I'll let you know when it finishes. You can restore by exporting your credentials and then running datomic restore-db.

$ export AWS_ACCESS_KEY_ID=blahhhh
$ export AWS_SECRET_ACCESS_KEY=flannnnn
$ bin/datomic restore-db s3://birdman.neo.com/backups/2014/02 datomic:dev://localhost:4334/birdman

Replace the last argument with your database connection string. If it is the dev transactor it needs to be running. I think SQL / DDB / other transactors need to be shut down during restore

Bestra commented 10 years ago

:shipit: !!! You sir, are a true gentleman. I'm reading over stuff right now. I'd love to see this before we drive up to Detroit.