mysociety / pombola

GNU Affero General Public License v3.0
65 stars 41 forks source link

Update people data and come up with maintenance process #875

Closed dracos closed 10 years ago

dracos commented 11 years ago

The JSON file is more up-to-date than any current dev or staging install I can see (e.g. Mr Nqabayomzi Lawrence Kwankwa); it needs loading in, and a process decided upon as to where updates to people data will be made (e.g. stop importing JSON file, update live db; maintain JSON file, perform db updates from it; etc.)

osfameron commented 11 years ago

the data is in pombola/south-africa/data/south-africa-popolo.json

I'll update this on staging now.

It makes sense to me to continue updating the .json as the master image at least for now -- as this makes it easy to import into popit also.

evdb commented 11 years ago

Have we decided that the south-africa-popolo.json is to be the canonical source? If so it might me worth spending some time creating a script to export a database to popolo JSON (or more likely rework the existing core_send_people_to_popit scripts).

My current reason for this is that I've previously imported biographies (#931) that are only in db and not in the popolo file, and now I need to do something similar for #885. There are lots of tools to help apply this sort of stuff to the database, but not direct into a popolo file. Perhaps loading the popolo, amending the database and then writing out the popolo again is the most pragmatic approach now, even though the ideal would be to have tools to manipulate the popolo directly.

Thoughts on this?

dracos commented 11 years ago

I had assumed that people would be using the admin interface to update details on the live site, once it exists, and thus the database would have to be the canonical source, the JSON only used to create an initial import for the live data. But until we have a live site, that the JSON file would be updated with new data specifically so there would be no confusion as to which data had been updated where in which database and you could just load in one file to have it up and running correctly to begin. So #931 could also have been done by manipulating the JSON file and reimporting.

878 and related (tidying up precisely what Kind a committee is) still needs doing to stop the initial import having needless duplicate, possibly incorrect, information in it. Then as long as the import process is fully documented, it doesn't really matter, until such time as modifications start being made by people using the admin interface...

evdb commented 11 years ago

931 should have been done onto the Popolo, it just slipped my mind as I was working on it. As #878 et co are probably best done using by editing the popolo file I'll look at the easiest way to get db changes back into it and then update it with the #931 changes, and the as yet not applied changes from #878

paullenz commented 10 years ago

PMG now have credentials to edit via the admin, staging will become live and is now canonical