Explore how to write query results into a Wikibase

[Daniel-Mietchen]

This is one of the topics for which I thought we had a ticket already and wanted to re-read it, but I could not find one right now, so opening this one.

The basic idea has several components, e.g.:

• writing down query (or even subquery) results (from Scholia panels or more generally) can help in various ways, e.g. with caching, addressing performance issues with complex queries/ trees/ popular properties, assisting with evaluating usage (e.g. by property, Scholia panel, or over time)
• writing them down in a Wikibase allows for annotation (e.g. with timestamp or query ID) and querying
• exploring this means we would need to think about
  • data models, e.g.
  • do we need that Wikibase to accept CSV, TSV or other WDQS result formats as data types?
  • do we want to model tabular data simply by row and column, with series ordinal, or in some other way?
  • what about non-tabular data - shall we always store it in its native format (perhaps with some screenshot or other representation) or as a table and then re-render?
    • what about media files from Commons that were part of the query results in an image grid?
  • licensing of that Wikibase/ individual results pages
  • CC0 makes the most sense, but this would limit doing things capturing image grids
  • automation, e.g.
  • tooling to edit that Wikibase
    • think QuickStatements or Wikidata Integrator or SPARQL CONSTRUCT queries
  • performance
  • in terms of parameters like
    • lag
    • bandwidth
    • size limits of query results
    • timeout settings of that Wikibase's SPARQL endpoint
    • whether and how to limit the user pool
    • a pragmatic approach here could be to make one of those Scholia-only
  • customization possibilities, e.g.
  • before rendering a Scholia profile, we could run a quick query on that Wikibase that checks whether all the panels relevant for the profile do render, and only shows those that do or that the user had specified (e.g. via URL parameters, as in #1114 )
  • checking for cases where a particular Scholia panel did not render at a given point in time would just be a SPARQL query over that Wikibase
  • privacy issues
  • if the query/ subquery results contain personally identifiable information
  • user contributions to query refinements could be tracked the usual wiki way

[Daniel-Mietchen]

Found an earlier thread on this in https://github.com/fnielsen/scholia/issues/67#issuecomment-438626312 .

[Daniel-Mietchen]

See also https://www.wikidata.org/wiki/Wikidata:WikiProject_Schemas/Subsetting .

[Daniel-Mietchen]

Thought a bit more about this, and think we should definitely be exploring SPARQL CONSTRUCT queries. Brief intro here.

[Daniel-Mietchen]

There are several properties for handling tabular data hosted on Wikimedia Commons: https://www.wikidata.org/wiki/Special:ListProperties/tabular-data .