search

europarl / open-data-beta-testing

European Parliament Open Data - Call for beta testers
https://europarl.github.io/open-data-beta-testing
34 stars 3 forks source link

API? #4

Open hejjoe opened 2 years ago

hejjoe commented 2 years ago

Will there be an API to automatically pull data from the EP, or how will it be possible to include new datasets in automated workflows?

andrea-perego commented 2 years ago

@hejjoe , data will be available from REST APIs once the European Parliament's Open Data Portal will be launched.

About "automated workflows", do you have in mind any specific use cases?

hejjoe commented 2 years ago

A case could be: Let me pull and reuse the data from the latest plenary session, as soon as it is available.

E.g. https://github.com/OpenParliamentTV/OpenParliamentTV-Parsers https://github.com/ungeschneuer/plenum_first_said

tttp commented 2 years ago

let me add ours to the list: https://github.com/TechToThePeople/parlparse ;) used for mepwatch.eu

the problem we face: 1) we don't know when the EP publishes a new rollcall, we can't even rely on the calendar, as there are some plenary days without rollcalls (eg mondays) 2) we need to scrape the EP website and there is no up to date computer readable list of MEPs as far as we know

@hejjoe, beside a good structure (and this repository goes in the right direction, well done), what is still unclear is:

andrea-perego commented 2 years ago

@hejjoe , @tttp , thanks for contributing these use cases.

About notifications, which mechanisms do you suggest? A traditional RSS / ATOM feed, LDN (https://www.w3.org/TR/ldn/), or other solutions?

@tttp , about your questions:

  • where are you going to publish this data (api/url)

They will be available via the API of the European Parliament Open Data Portal.

  • is there a metadata explaining what is updated where (eg new MEP, new plenary)?

We plan to release a new version of the relevant datasets whenever changes occur. These datasets will be documented by using DCAT, but we don't plan to describe the actual list of changes.

Do you have any suggestion on how to deal with this?

  • is there a plan to give early warnings on data schema changes?

Thanks for raising this issue. The schemas will be maintained on GitHub, where changes will be documented before the release of new versions.