Add seed data and instructions

CodeWithAloha / uipa

Helping submit, track, and share public records requests in Hawaii

http://uipa.org

MIT License

10 stars 6 forks source link

Add seed data and instructions #80

Closed russtoku closed 3 months ago

russtoku commented 3 months ago

Preliminary information to seed a development database. This should be enough to anyone started and provide a base to move forward from.

I'm still working on a Python script to create a category.json fixture and CSV file for uploading public body data from the CSV extract from the production UIPA.org website.

yenhtran commented 3 months ago

Thank you @russtoku for the great instructions!

russtoku commented 3 months ago

Apologies for the large number of changes:

Re-write of Seeding.md
Renamed the fixture files with a timestamp
Added some utility programs to help with preparing a public body CSV file that can be uploaded via the Admin website.

russtoku commented 3 months ago

These are working now:

Load the classification, jurisdiction, and foilaw fixtures.
Load the test categories and test public bodies.
Load the full list of categories from the 03-15-2024 extract of public bodies from UIPA.org.
Only 118 out of 201 public bodies can be loaded from the fixed 03-15-2024 extract of public bodies from UIPA.org. This should be enough to work with.
Two shell scripts can be used to "reset" the database and search engine indexes.

I'd like to suggest that we merge things at this point to give people something to work with. The seeding of all 201 public bodies will take a few more days. If I finish that before this is merged, I'll add it.

russtoku commented 3 months ago

I can't find a permanent fix for loading public bodies from a CSV file. I've updated the Seeding.md document to point this out. Loading abou 113 public bodies should enough to start developing with.

I'd like to see more people getting involved so hopefully the seeding of the database helps make the set-up process easier.

russtoku commented 3 months ago

I discovered that deleting and reloading the public bodies from a CSV file several times will eventually load all 201 public bodies correctly. I've this to a note in Seeding.md.