Migrate from popit-django to django-popolo / multiple-django-popolo-sources

[mhl]

PopIt is being deprecated, so it no longer makes sense for write-it / WriteInPublic to depend on PopIt instances as sources of data, or for it to depend on the deprecated PopIt client, popit-django. It makes more sense now to move to a model where data sources are all URLs of Popolo JSON, and to use the django-popolo project to store the data from such Popolo JSON sources. multiple-django-popolo-sources can be used to keep the data from each of these sources distinct.

(There is also the PopIt 2.0 / NG project from Sinar now; so long as that exposes an endpoint similar to PopIt's export.json, this can be still be used - we should check that, and make a feature request if not.)

This pull request migrates from using popit-django and PopIt to using django-popolo, multiple-django-popolo-sources and Popolo JSON URLs instead.

Notes for reviewers / self review

This is a large pull request, so at the request of @chrismytton I'm going to split it into smaller ones targetting a branch called reviewed-migrate-to-django-popolo, since it still needs to be deployed in one go. Many of the commits after the data migration (i.e. after "Remove a now unneeded popitapiinstance field") could be squashed into one commit which represents updates to the code and tests for the new models, but this would make it much harder to review.

Another consideration is that there are a lot of instances in writeinpublic.com - I've tried the data migration on various sources locally, but:

• I don't know which live instances are actually used or are particularly important, we'd want to at least check that those are migrated fine.
• I think it would be a good idea to set up a staging site with a replica of the live data (with DEBUG=False, but in test mode) so that we can try it on real data.

  Sub-Pull Requests

• Part 1: add django-popolo data in parallel https://github.com/ciudadanointeligente/write-it/pull/1193
• Part 2: remove references to old popit-django models https://github.com/ciudadanointeligente/write-it/pull/1195
• Part 3: update all the code and tests https://github.com/ciudadanointeligente/write-it/pull/1196

[coveralls]

Coverage increased (+0.07%) to 98.518% when pulling f3523442f82cdc2e7ddb915fb33b0ba63ce5206e on migrate-to-django-popolo into 7d7615420c3f54cf0b743be58226ec277514381f on master.

[coveralls]

Coverage increased (+0.07%) to 98.518% when pulling 07352bd067924be2391b07c480caa18e641fd157 on migrate-to-django-popolo into 7b1b98446e33c6ca2c3f05389c376c2f1db6946d on master.

[coveralls]

Coverage increased (+0.07%) to 98.518% when pulling 500c1db4f3016f887c91a3f304cb359c50383853 on migrate-to-django-popolo into 7b1b98446e33c6ca2c3f05389c376c2f1db6946d on master.

[coveralls]

Coverage increased (+0.07%) to 98.518% when pulling 4c5fa8d9b03fd8976f5c4073bdf0b1e4927ec57c on migrate-to-django-popolo into 7b1b98446e33c6ca2c3f05389c376c2f1db6946d on master.

[coveralls]

Coverage increased (+0.07%) to 98.518% when pulling 27633824edc9b39b4cdd63ec55035beb23e15c59 on migrate-to-django-popolo into 7b1b98446e33c6ca2c3f05389c376c2f1db6946d on master.

[coveralls]

Coverage increased (+0.07%) to 98.518% when pulling 121164626035f5074afde5f891517c769c303d0d on migrate-to-django-popolo into 7b1b98446e33c6ca2c3f05389c376c2f1db6946d on master.

[coveralls]

Coverage increased (+0.07%) to 98.518% when pulling ae241947c44fd1382cb74ed44d8faba35256510d on migrate-to-django-popolo into 7b1b98446e33c6ca2c3f05389c376c2f1db6946d on master.

[coveralls]

Coverage increased (+0.07%) to 98.518% when pulling a8a4f80736c417b4a51960a0fc026e202a5d2b51 on migrate-to-django-popolo into 7b1b98446e33c6ca2c3f05389c376c2f1db6946d on master.

[coveralls]

Coverage increased (+0.07%) to 98.518% when pulling d212d48e6d58cc78e6bc1f81ed7fa6ba4d753e64 on migrate-to-django-popolo into 7b1b98446e33c6ca2c3f05389c376c2f1db6946d on master.

[coveralls]

Coverage decreased (-74.8%) to 23.627% when pulling dc98beae27662a0198656ad29cd3f7bfdd8598c6 on migrate-to-django-popolo into 7b1b98446e33c6ca2c3f05389c376c2f1db6946d on master.

[coveralls]

Coverage increased (+0.07%) to 98.52% when pulling 0ecc77eec82543e1bf7b2c109334a735c51203e5 on migrate-to-django-popolo into 7b1b98446e33c6ca2c3f05389c376c2f1db6946d on master.

[coveralls]

Coverage increased (+0.07%) to 98.52% when pulling 0ecc77eec82543e1bf7b2c109334a735c51203e5 on migrate-to-django-popolo into 7b1b98446e33c6ca2c3f05389c376c2f1db6946d on master.

[coveralls]

Coverage increased (+0.07%) to 98.52% when pulling 2a6c2a249e042e796a9a681117ce537fa4290d23 on migrate-to-django-popolo into 7b1b98446e33c6ca2c3f05389c376c2f1db6946d on master.

[coveralls]

Coverage increased (+0.07%) to 98.52% when pulling e22e88936d7fc551ffbef327f247903185215e11 on migrate-to-django-popolo into 7b1b98446e33c6ca2c3f05389c376c2f1db6946d on master.

[coveralls]

Coverage increased (+0.07%) to 98.52% when pulling 5afb702e1e8a30af347c8edd8cfaede74db454d0 on migrate-to-django-popolo into 7b1b98446e33c6ca2c3f05389c376c2f1db6946d on master.

[coveralls]

Coverage increased (+0.07%) to 98.52% when pulling 66160ecda2662fcee991b969cbd6489224e24b30 on migrate-to-django-popolo into 0830395873eb5c6c6b9b5915219a3b985008b427 on master.

[coveralls]

Coverage increased (+0.07%) to 98.52% when pulling 1c63a1509fbb45bce508032a8ee27f6f2e6ef744 on migrate-to-django-popolo into 0830395873eb5c6c6b9b5915219a3b985008b427 on master.

[coveralls]

Coverage increased (+0.07%) to 98.52% when pulling d579dabc0e70c5490d712b50823e8e82fdc150d8 on migrate-to-django-popolo into 0830395873eb5c6c6b9b5915219a3b985008b427 on master.

[coveralls]

Coverage increased (+0.07%) to 98.52% when pulling 3c708288c87628ebd56fd0e1e8edaf9fbfea4079 on migrate-to-django-popolo into 0830395873eb5c6c6b9b5915219a3b985008b427 on master.