search

opengovfoundation / madison

Madison is a platform for lawmakers to share legislation with their citizens, allowing the community to add comments and suggest improvements.
GNU General Public License v3.0
690 stars 108 forks source link

Migrate Annotations from ElasticSearch to Mysql #162

Closed cmbirk closed 10 years ago

cmbirk commented 10 years ago

We need to stop storing the annotations in ElasticSearch to make use of the Eloquent ORM. We'll also need to account for storing comments on annotations ( which are currently just stored in the annotation JSON ).

cmbirk commented 10 years ago

Annotator posts a json payload to /api/docs/:doc_id/annotations . This is handled by the AnnotationApiController, and the annotations are stored as their raw json in ElasticSearch.

Any comments that have been posted on the annotations are simply stored as a json array attached to the annotations.

Instead of storing the annotations as raw json in ElasticSearch, we should store these in the mysql database by creating migrations for the annotations and their comments. The AnnotationApiController should still index the annotations in ElasticSearch for search purposes, but they should loaded from mysql for everything else. This allows us to use the Eloquent ORM that Laravel provides and makes it much easier to access the annotations.

coogle commented 10 years ago

I just pushed this, which should implement this feature.

https://github.com/opengovfoundation/madison/tree/issue162

Note, I am presently NOT changing the read methods to not use elastic search, I am only also saving the annotations and their comments to the database with the models associated with them to do stuff. Let me know how much additional stuff you want.

cmbirk commented 10 years ago

@coogle would you change the read methods to use the DB? I'm planning on using ES for search only to reduce the complexity of the more important functions. ie. If an annotation is broken in one place, I know it's broken in another. I'm not putting as big an emphasis on search just yet.

coogle commented 10 years ago

Okay I think we're done with this now as of my latest pushes to the issue162 branch. The annotation model is completely re-done to be DB based, and the annotations are being pushed to ES as necessary as a secondary store.