Swedish, 1.3 rebase - Githubissues

AutomCoding commented 1 year ago

Replacement for #580 as I could not figure out how to change the target branch of a pull request.

EeveesEyes commented 1 year ago

Just a quick status update: we are discussing how to handle the Swedish translation, since we use a separate elastic search index for each language. This means, for example, that the French version of the openart browser will only contain artwork provided by wikidata in French. To add a translation, we would either have to do another crawl of the Swedish Wikidata or change our index translation strategy by somehow decoupling it from the available data. We are not sure what would be best yet.

AutomCoding commented 1 year ago

There are 106607 instances of [subclass of artwork] located in Sweden and most have a label in English. I assume the language overlap to be greater outside of Sweden, but those queries are too heavy to run. So a quick interim solution could be to reuse that list as that would cover 60 %.

Language	Without label	Percentage
Swedish	131	0.12 %
English	41911	39 %
German	103161	97 %
Italian	106481	99 %
Spanish	105909	99 %
French	105963	99 %

EeveesEyes commented 1 year ago

Sorry, that was a bit misleading, plus I was wrong :laughing: . The location of the artwork doesn't matter. Actually the crawl fills unavailable translations with the highest ranked language available. In example: if Swedish is not available for a particular artwork, english is chosen. If that is unavailable, too, french is next. This behaviour is described in our developer guide. So it would be good to know how many swedish labels would be available, but I'm finding it hard to get this query working. To get a list of all crawled types, see contstans.py

AutomCoding commented 1 year ago

After some research I found a way to get a list of all 18412 subclasses of artwork in just about one second. There are 3481176 items that are instances of one of them (runtime one minute). I was not able to look up their labels and filter, but this gets through one tenth of them before crashing:

SELECT DISTINCT ?artwork WHERE {
  SERVICE gas:service {
    gas:program gas:gasClass "com.bigdata.rdf.graph.analytics.BFS";
      gas:in wd:Q838948;
      gas:linkType wdt:P279;
      gas:traversalDirection "Reverse";
      gas:out ?type.
  }
  ?artwork wdt:P31 ?type.
  ?artwork rdfs:label ?label.
  FILTER((LANG(?label)) = "sv")
}

EeveesEyes commented 1 year ago

I ran our crawler (without actually crawling anything) to get some statistics without running into timeouts. Based on our current type selection there are approx. 94k items with labels in Swedish. The remaining ~670k artworks would appear in other languages / without label in case they don't have one in any of the supported languages.

Click to expand table

hochschule-darmstadt / openartbrowser

Swedish, 1.3 rebase #581