search

chihacknight / chicago-elections-archive

Explore precinct-level results from historical Chicago elections
https://chicago-election-archive.netlify.app
MIT License
0 stars 0 forks source link

Geojson geodata #9

Closed nofurtherinformation closed 1 month ago

nofurtherinformation commented 2 months ago

Summary: This PR relocates the existing geojson data from the current Linode server to an s3 compatible Cloudflare R2 bucket, and swaps the map frontend to use Geojson instead of map tiles.

Testing:

Details:

This PR also includes some changes to how env/metadata is passed from the build process to the frontend javascript runtime. Passing data into the nunjucks template seems more conventional than head properties, but there does not seem to be a great way to pass environment variables from 11ty to Solid.

This PR also dynamically adds sources on layer update rather than at build time. The reason for this is that all the geojson sources would load initially, which is like 20mb of data.

Last, this PR includes a script to pull existing geojsons and upload them to the specific bucket, which we can use wherever the final data ends up being.

In the future, we'll want to centralize this bucket (or a bucket) as the final home for the data, and point the CSV data there, too. For now I think we can fragment the data a bit for now while we migrate stuff over and rethink storage formats.

derekeder commented 2 months ago

@nofurtherinformation nice! Chi Hack Night does not currently pay for either Cloudflare or AWS S3. Considering this data is pretty small (~20mb) and doesn't change much, any downside to committing the files to a file in the repo itself and referenced there?

nofurtherinformation commented 2 months ago

@derekeder Not much downside! All the standard caveats about github data storage etc etc, but this data likely won't grow that much (1mb or so per year?) -- I'll pull the data into the site's public files

nofurtherinformation commented 2 months ago

This PR now saves geojsons to output/geojson, which builds into data/geojson/.... I've allowlisted output/geojson in gitignore, but kept the output folder ignored by default, in case there are some other data processes going on that may be bigger than we want.