openaustralia / jacaranda-righttoknow

a watchful tree and slack messenger to keep you informed of the use of Right To Know
0 stars 0 forks source link

Make a generic Alaveteli jacaranda that can be used by any of the instances #6

Open equivalentideas opened 7 years ago

equivalentideas commented 7 years ago

I spoke a bit about this idea at TICTeC in April https://www.slideshare.net/mysociety/automating-feedback-the-answer-for-small-civic-tech-teams-luke-bacon-openaustralia-foundation

Some thoughts:

auxesis commented 7 years ago

I did 10 minutes of research this evening, and I think it'll be extremely simple to implement (if you ignore i18n).

We can probably refactor the RightToKnow class to set a host to connect to based on an environment variable.

Pretty much all of the existing counting table of varieties lookups should work, regardless of the site.

equivalentideas commented 7 years ago

there's an issue with how jacaranda scrapes information about requests off the front end openaustralia/righttoknow#673 it currently relies on an assumption about how many requests there are a fortnight on the site to work. The different Alaveteli sites have wildly different frequency of requests, and it changes, so this needs to get smarter or we need a more reliable way to get the information from the sites.

I think that's still a problem, because we don't currently know of any way to get an accurate number for the lookups from whatdotheyknow.com . The bug that prevents the current method from working has been effectively closed as 'wont fix': https://github.com/mysociety/alaveteli/issues/456#issuecomment-322976225

I've asked the Alaveteli maintainers if they know another way, but so far they don't.

One thing we could do is expand the requests api which currently includes 'hides_by_week' to include 'success_by_week', and 'new_by_week'. We could add an annotations API that did the same thing. Then we could just use the API rather than scrape the front.

... And then we could update Right To Know so we get all these goodies 😆 https://github.com/openaustralia/righttoknow/pull/686 .

🤔