search

TBFY / knowledge-graph-API

This repository is dedicated to the core API of the TheyBuyForYou project built on top of R4R (https://github.com/TBFY/r4r)
2 stars 1 forks source link

Mass data access #32

Closed matejpos closed 4 years ago

matejpos commented 4 years ago

In order to perform data analysis, I need to download all the necessary data from KG (and continue downloading in cronjob to keep analysis updated). I'm encountering two major issue related top current api, when trying to collect the data from the KG:

(1) For every tender, I need to make lots of requests in order to collect all data associated to a specific tender, which doesn't seem like an optimal way of collecting data. My question would be: would it be possible to request let's say 100 tenders with all associated data in an ocds-like format? If not, would it be possible to reduce the number of requests, for example, to submit a list of tender ids and retrieve all associated tender data? Or even to perform a custom sparql query on KG?

(2) It seems like the api is offering just a limited set of tender parameters from the KG. How would it be possible to access all necessary data? Possibly directly from KG? Custom queries?

A super useful feature would be would be the possibility of a mass download from the KG. And one more question: would it be possible to sort tenders by country - or I need to do it by reading ocid and then from the ocds ids list figure out the country origin?

Thanks Matej

fyedro commented 4 years ago

Periodic data dump on data-sources repository (https://github.com/TBFY/data-sources)