search

openelections / openelections.github.io

Entry point into Openelections project status and data
MIT License
12 stars 11 forks source link

A Better openelections.net #63

Open dwillis opened 5 years ago

dwillis commented 5 years ago

The openelections.net site is badly in need of a refresh. In particular, it doesn't provide updated information about what data we have, what states are in progress and how to use our data collections. Some things I'd like to propose:

I'd love to hear comments on these ideas as well as any other ideas you might think useful. There's a Google doc for brainstorming: https://docs.google.com/document/d/1I90PupYP_5R5rqxnzxbG7hr-uaH2gRo7I6iQhFH9E48/edit#

miklb commented 5 years ago

I mentioned on Twitter that I'm interested in contributing on any Jekyll/plumbing issues.

I'm still not 100% on where GitHub Actions fall with organizational repos but something could be wired together to regularly force a build to pull in new data and rebuild. That would help address "lessen need to manually update" which could include pulling from Sheets, GitHub API and any other data sources.

What type of DB is being used now that you're interested in moving away from?

dwillis commented 5 years ago

@miklb Yeah, I think GitHub Actions might be a good way to go. It would be interesting to see if we could use it to update state-based pages with repository activity. The only database that the site uses (and it uses data exported from it at that) is a Postgres database of election metadata - details about when elections have been held and what offices are on the ballot. I can produce a dump of that and probably will convert it to SQLite.

miklb commented 5 years ago

I've run Jekyll on Travis CI to run a RAKE task to pull JSON via API request and commit it back to the repo in the _data directory. Then "dynamically" populate pages from that. I have plans to do that in an Action instead and believe it would work here as well.

Basically a cron task to pull in new data every X hours/days.

edit Jekyll on GitHub pages specifically.

dwillis commented 5 years ago

I think my preference would be to use Google Sheets as a backend for metadata and GitHub's API for information about specific activities. That likely means coming up with a structure that works well for providing summary information about an election in every state as well as state-specific information.

miklb commented 5 years ago

Were you thinking via Javascript then or still pulling in the data and storing as JSON?

dwillis commented 5 years ago

I think pulling JSON from Google Sheets using Javascript is probably the way to go, although I'm also thinking about SQLite as a backend that we could maintain and publish (maybe even multiple SQLite dbs).

miklb commented 5 years ago

If you go SQLite you could use https://github.com/simonw/datasette as an api. Or perhaps I’m over thinking it.

dwillis commented 5 years ago

No, I think that's a possibility for sure.

miklb commented 4 years ago

I have some bandwidth to contribute still if there are any tasks I can help with. Should I just look at open issues or wait for you to gather more feedback?

dwillis commented 4 years ago

@miklb I think what I'd like to do is to gather ideas about what sort of things we should have on the site. Like, what should appear on a state page? Which activity should we display on the home page? Maybe a google doc would kick things off: https://docs.google.com/document/d/1I90PupYP_5R5rqxnzxbG7hr-uaH2gRo7I6iQhFH9E48/edit

miklb commented 4 years ago

Excellent idea. I’m not sure I have answers to those questions, but can help look for them.

nbdavies commented 4 years ago

Here's something I mocked up, for how elections/results could be presented interactively (using Vue.js for example), and hosted statically wherever: https://nbdavies.github.io/openelections-website-wi/

The election details are pulled from JSON on Github (the same data that's in @openelections/openelections-metadata).

Source here: https://github.com/nbdavies/openelections-website-wi/tree/gh-pages

That doesn't tackle the problem of showing what's been updated recently, or what's available/converted vs what isn't. Those kinds of things might require a Github API key, and might not be amenable to a frontend-only site.

mroswell commented 4 years ago

@nbdavies I like the concept. The filters seem to behave in a someone wonky fashion. Took me a long time to figure out that they were "or" filters. Maybe best to add checkboxes to make that clear.

dwillis commented 4 years ago

@nbdavies I agree - I like this, too. I wonder if this could serve as the basis for state pages going forward.