Create a new GitHub organisation, split repositories, and refactor

[wheresalice]

We currently use a single repository which has grown organically, and is now serving the purpose of both loading and preparing data as well as a very specific use-case of the data - serving a website. We should create a separate GitHub organisation for this project, split the current code into a website repo and a data repo, and refactor the data repo to be able to run in Docker more easily. We should also store the geojson data in the website repo.

Motivation

As a frontend developer
I want to be able to work on the website without processing the data myself
So that I can deliver value more quickly

As a data analyst
I want to be able to quickly get clean data in my tool of choice
So that I can deliver value more quickly

Specification

• Create a new GitHub organisation
• Move map/static into a new repository in the new organistion, along with the wards.json file, and docs on running it
• Refactor the rest of this repository to have a single Makefile, single data directory, and single scripts directory

Downstream Impact

This may require minor changes to how the current Datasette deployment is performed

[kimadactyl]

Hmmm I'm totally on the fence about this right now as I was thinking of this more as a toolkit than an endpoint in itself. My counter user story would be:

As an activist working to understand stop and search
I want to have an easy starter kit for understanding, manipulating and presenting stop and search data
So that I can work with people in my area to understand police oppression

Maybe let's go over it on Tue :)

[kimadactyl]

If we went my route I think I'd be tempted to:

• Have one big Makefile that does things like: downloading data, running servers, basically everything in one.
  • data directory with subdirectories for police data, wards, etc.
  • map -> chloropleth
  • map outputs builds to docs directory so it can be used with github pages. idea would be to eventually have this overlay all the other data too when we figure out how
  • what's current in the root directory -> search_browser or something
  • root folder has core info about the project, advertises it as a toolkit with some howtos, etc
  • maybe one big setup task that does all the data downloading and dep installation based on some parameters for you (i.e. region)
  • maybe rename it the stop and search SDK if that's not too naff? :)