update site's data

[loicjaouen]

The web app is still talking to unil's server because:

• the data used to be at unil
• the data has been moved to DaSCH but performance at first were not matching the webApp expected reactivity
• some requests are too complex to be expressed in gravsearch (the DaSCH query language) and request directly the triple store with SPARQL queries in order to build cache

But now the data is at the DaSCH and the one at unil is getting outdated.

---

Status and TODOs:

• [x] cache updater
  • [x] stop the cache updater to run automatically
  • [x] adapt it to access fuseki (rather than graphdb)
  • [x] run it by hand
• [x] web app
  • [x] change the config to tackle DaSCH's server rather than Unil's
  • [x] split up problematic requests in smaller ones
  • [x] bundle and release

[loicjaouen]

testing DaSCH speed

For common requests, simple gravsearch queries and getting a resource by ID, it is fine now.

For the complex queries, running them on the fly (rather than pre-ran while building cache) take more than 30s for the map for example.

The cache is still needed.

complex queries

The queries mentioned there are using sparql features such as group by, group_concat, sample, min and max, count (get_calendar, get_places, get_works) and graph building by requesting data from different resource classes (get_authors).

It is either some histogram request, like sampling the number of representation for a given year, or finding persons but not with the resource Person being the root of the network exploration (authors are persons that have works mentioning them as authors).

In these cases, while sparql does it for us, without sparql, we have to dump the data to iterate over it and increment our counters.

Each page of a data dump takes between 500 to 800 ms, dumping the resource class Representation, close to 2'000 resources, 40 pages, takes more then 30 seconds => we need caching.

[loicjaouen]

the cache builder

the cache builder is an automated way to send the queries and format the result into suitable data for the webApp.

We can't make it run automatically when accessing the DaSCH because the triple store is not exposed.

For now we can still run it by hand.

TODO:

• [x] stop the cache updater to run automatically
• [x] adapt it to access fuseki (rather than graphdb)
• [x] run it by hand

[loicjaouen]

possible evolution: rework the sparql queries and turn them into gravsearch queries and code to aggregate properties' value.

[loicjaouen]

DSP API Calls

this is a configuration issue:

• [x] change the config to tackle DaSCH's server rather than Unil's

[loicjaouen]

stopped updater on the server

[loicjaouen]

the rest of the site is working at an acceptable speed except the calendar page listing the representations for a given year, it is slow, about 50s and puts the whole server on hold, waiting for this request to complete.

PREFIX knora-api: <http://api.knora.org/ontology/knora-api/v2#>
PREFIX theatre-societe: <http://api.ls-prod-server.dasch.swiss/ontology/0103/theatre-societe/v2#>

CONSTRUCT {
    ?representation knora-api:isMainResource true .
    ?representation theatre-societe:representationIsBasedOn ?work .
    ?work theatre-societe:workHasTitle ?playTitle .
    ?representation theatre-societe:representationHasPlace ?place .
    ?place theatre-societe:placeHasName ?placeName .
    ?representation theatre-societe:representationHasDate ?date .
} WHERE {
    ?representation a knora-api:Resource .
    ?representation a theatre-societe:Representation .
    ?representation theatre-societe:representationIsBasedOn ?work .
    ?work theatre-societe:workHasTitle ?playTitle .
    ?representation theatre-societe:representationHasPlace ?place .
    ?place theatre-societe:placeHasName ?placeName .
    ?representation theatre-societe:representationHasDate ?date .

    FILTER(knora-api:toSimpleDate(?date) = 'GREGORIAN:1714-1-1:1714-12-31'^^<http://api.knora.org/ontology/knora-api/simple/v2#Date>)
}

ORDER BY ?date

[loicjaouen]

• [x] split up problematic request

[loicjaouen]

deployed