search

ec-doris / kohesio-backend

APIs serving Kohesio's frontend
https://kohesio.ec.europa.eu
6 stars 2 forks source link

Expose Kohesio query endpoints #88

Closed madewild closed 3 months ago

madewild commented 2 years ago

Currently https://query.linkedopendata.eu/ is the only endpoint we can query from the browser. Following the logic of separation, when we have the new url we should create:

https://query.kohesio.ec.europa.eu to query the prod index https://dev.query.kohesio.ec.europa.eu to query the dev index

While of course keeping the live endpoint of Wikibase. :)

madewild commented 2 years ago

Will also help with the "edit graph" button and the problem we had for France...

D063520 commented 2 years ago

Open qanswer sparql endpoint ..... with UI

madewild commented 2 years ago

In the end the URLs would be like:

https://kohesio.ec.europa.eu/query https://kohesio.acceptance.ec.europa.eu/query https://kohesio.test.ec.europa.eu/query https://kohesio.development.ec.europa.eu/query

Would that be a problem for the application? Difficult to ask for subdomains on Europa... We can reuse what we have on https://dev.qa.linkedopendata.eu/user/endpoint?kb=eu&user=commission (without password)

D063520 commented 2 years ago

Hi,

so I think the only thing that we need to do is to point:

https://kohesio.acceptance.ec.europa.eu/query

to

the qanswer instance at

/user/endpoint?kb=eu&user=commission

@raphdom if you give me very rough instructions I can try myself

IMPORTANT: @madewild I do not recommend to expose the query endpoint of PROD. The reason is that someone could run a heavy query. This would slow down the application. If we want to check something we can always check on acceptance. no?

madewild commented 2 years ago

For us yes, but the point is to offer an open query endpoint to all users, journalists etc. Currently we have https://query.linkedopendata.eu/ but:

  1. it is over the entire KG and not just Kohesio data (frozen index)
  2. there is a timeout of 60 seconds
  3. it is sitting on an EC2 which does not scale

My understanding is that the cluster could handle the extra load. @raphdom do you confirm? @athollard what is your view? is the query endpoint a core requirement for transparency and openness?

raphdom commented 2 years ago

@D063520 I have setup the test environment for the qanswer application with /qa https://kohesio.test.ec.europa.eu/qa But seems that something is not working and the page is not loaded. Do you think that will work only with /query? Maybe you should change something in the application to map this url? Let me know...

madewild commented 2 years ago

Caution @raphdom: /query and /qa are two different things! See #123 for the /qa part...

raphdom commented 2 years ago

Ah okay, different things for the same application. The app should take care of this, on my view, but we can discuss this.

madewild commented 2 years ago

To clarify, currently we have two distinct services:

These are different applications. Now on Kohesio, QAnswer will handle both. So maybe we could have a single URL for both, @D063520 what do you think?

AThollard commented 2 years ago

The query endpoint is not mandatory but we have mentioned it as a service in both business case and project charter. So I think it would be good to keep it as a separate service. Also because it's a service out of the box from the Wikibase if I am not wrong and so a good 'hook' for Wikimedia Germany to promote Wikibase related services. If it's a timing and scope issue it can be postponed to version 1.2. As for version 1.1 the focus will not be on query service but on datasets exports and availability. We could then create a momentum for version 1.2 with both query service and QAnswer natural language queries? What do you think?

madewild commented 2 years ago

The two services above will remain in any case, it's a matter of adding someting dedicated on the Kohesio website or not. https://query.linkedopendata.eu/ comes out of the box from Wikibase/WMDE but if we add on kohesio.ec.europa.eu it would be a service provided by QAnswer like we currently have on https://dev.qa.linkedopendata.eu/user/endpoint?kb=eu&user=commission

AThollard commented 2 years ago

I don't get the point on embedding the query endpoint in QAnswer, is it to restrict the scope to Kohesio only and not querying the full KG? Or is it to promote QAnswer as a global service? Let's discuss this briefly so you enlighten me, in any case not a priority for the launch.

madewild commented 2 years ago

After discussing with @AThollard we will postpone this to 1.2

madewild commented 3 months ago

I think this is no longer needed