[Document] Summer Hackathon June 2017

[OriHoch]

This issue is the central point for all things related to the Summer Hackathon

current status of the Open Knesset project

overview

In the past Open Knesset project volunteers were focused mostly around the Open Knesset web-site, trying to make it work and show data about the Knesset. This is of course still the end-goal, but we decided to split the effort into 2 related but separate projects with sub-repositores.

sub-projects / repositores

• Open Knesset website (repo) - the website (both frontend and backend), and currently also a lot of scraping and processing code which we intend to move / refactor to other repositories (see below)
• Knesset Data - handling of the data - scraping or getting it from the Knesset APIs, parsing, converting, processing and exposing via API or DB. This project is split into 4 repositories:
  • knesset-data (repo) - the main repository, contains issues, documentations and details which are relevant across all the sub-projects
  • knesset-data-python (repo) - low-level python module which handles the scraping and fetching of the source data
  • knesset-data-datapackage (repo) - compiles the source data into a datapackage which conforms to the frictionlessdata standards
  • knesset-data-django (repo) - handles higher-level processing, statistial analysis, storing data in the Open Knesset DB and exposing it via API / django modules

what's up?

• New Knesset APIs - Knesset publsihed new APIs which they are pushing us to adopt. They are starting to phase out the old APIs and some stuff stopped working already. (related issue, another related issue).
• Legislature data - with the new API, the Knesset also has a new legislature website. Now, they stopped updating or have some bugs on the old website which Knesset don't intent to fix. So, we have to push forward the getting of legislature data from the new APIs. (related issue).
• Knesset-data pipeline - need to improve and add features to the knesset-data repositories, specifically, everything relating to syncing and updating data. We would like to use the datapackage-pipelines framework which is used by other Hasadna projects (like open budget). (related issue).
• there are many other bugs and features, see below about how to contribute and where and how issues are documented

how to help / where to start

• Backend / Python Developers

  • start with the knesset-data contribution guide
  • find a relevant issue to work on, see comments below for some ideas. (best to start with knesset-data-python, but knesset-data also has some good issues)
  • read the relevant repo README / CONTRIBUTING files for how to install and setup the development environment
  • generally, all repos follow very standard Python project flow -
  • project is a python module configured in the setup.py file, so you can run pip install -e . to install it
  • every project has automated tests and CI, you can check the .travis.yml file to see what we run
  • usually, projects have bin directory with some helpful bash scripts, like run_tests.sh

• Frontend / Fullstack / Web Developers

  • Due to the recent changes in repositories and focus of the project, we have less web development, but you have some options -
  • You can try to work on the Open-Knesset website itself, but I have to warn you that it will be difficult.. We use very old Django version, and code is a mess. But - if you are brave enough, go for it!
  • work on the knesset-data projects, just that you won't have much web development there, it's mostly backend python stuff, see the notes for backend / python developer above.
  • Perhaps you could have an idea for a new web dashboard interface, but it's not something we planned yet.
  • work on a new Open Knesset website which will work with Knesset-data directly. But, also here, we don't have a plan yet.

[OriHoch]

Issues

• all help wanted issues across all the relevant repositories