search

geogeeks-au / WAGeoMediaStatements

Build a site displaying geolocated media statements from the website https://www.mediastatements.wa.gov.au/
0 stars 0 forks source link

Object Persistance #2

Open Mappboy opened 8 years ago

Mappboy commented 8 years ago

Currently writing directly out to json. I am thinking of writing to a postgis table. I have tentatively added a pipeline for writing to postgis but would still need to write serializers for our item. Any other ideas ?

keithamoss commented 8 years ago

Write to PostgreSQL and store it as a single JSON blob per release for now? Then use magic JSON querying to do what we want with it after.

Only thing I'd store separately is the URL that was crawled. Then we can go back and get info about the Minister when we start to care about that.

Mappboy commented 8 years ago

Cool although I might store the location data separately as it's a one to many relationship. Currently for each statement I am writing multiple features for each point, however it might make more sense to write them as a multipoint ? Secondly having them in there own table might make it easier to process the statements again later with retrained models.