visualize-admin / visualization-tool

The tool for visualizing Swiss Open Government Data. Project ownership: Federal Office for the Environment FOEN
https://visualize.admin.ch
BSD 3-Clause "New" or "Revised" License
31 stars 3 forks source link

perf: Missing caching per cube iri for some queries #1651

Closed bprusinowski closed 3 months ago

bprusinowski commented 3 months ago

Fixes https://github.com/visualize-admin/visualization-tool/pull/1599#issuecomment-2205133720

vercel[bot] commented 3 months ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
visualization-tool ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jul 3, 2024 7:22am
bprusinowski commented 3 months ago

@Rdataflow this PR should hopefully fix the missing caching by cube iri queries 👍

Rdataflow commented 3 months ago

@bprusinowski there still some cases left... IIUC it occurs with maps

thanks for fixing this soon :+1:

CONSTRUCT { ?dimensionIri rdf:first ?unversionedValue . ?unversionedValue schema:name ?name ; schema:alternateName ?alternateName ; schema:description ?description ; schema:identifier ?identifier ; schema:position ?position ; schema:color ?color ; geo:hasGeometry ?geometry ; schema:latitude ?latitude ; schema:longitude ?longitude . } WHERE { { #pragma evaluate on SELECT ?dimensionIri ?versionedValue ?unversionedValue WHERE { VALUES ?dimensionIri { https://environment.ld.admin.ch/foen/nfi/unitOfReference } https://environment.ld.admin.ch/foen/nfi/nfi_T-changes/cube/2024-1 cube:observationConstraint/sh:property ?dimension . ?dimension sh:path ?dimensionIri . ?dimension schema:version ?version . ?dimension sh:in/rdf:rest/rdf:first ?versionedValue . ?versionedValue schema:sameAs ?unversionedValue . } } UNION { #pragma evaluate on SELECT ?dimensionIri ?versionedValue ?unversionedValue WHERE { VALUES ?dimensionIri { https://environment.ld.admin.ch/foen/nfi/unitOfReference } https://environment.ld.admin.ch/foen/nfi/nfi_T-changes/cube/2024-1 cube:observationConstraint/sh:property ?dimension . ?dimension sh:path ?dimensionIri . FILTER NOT EXISTS { ?dimension schema:version ?version . } ?dimension sh:in/rdf:rest/rdf:first ?versionedValue . BIND(?versionedValue as ?unversionedValue) } } UNION { { SELECT DISTINCT ?dimensionIri ?versionedValue ?unversionedValue WHERE { { #pragma evaluate on SELECT ?observation WHERE { VALUES ?dimensionIri { https://environment.ld.admin.ch/foen/nfi/unitOfReference } https://environment.ld.admin.ch/foen/nfi/nfi_T-changes/cube/2024-1 cube:observationConstraint/sh:property ?dimension . ?dimension sh:path ?dimensionIri . ?dimension schema:version ?version . FILTER NOT EXISTS { ?dimension sh:in ?in . } https://environment.ld.admin.ch/foen/nfi/nfi_T-changes/cube/2024-1 cube:observationSet/cube:observation ?observation .

        }
      }
      VALUES ?dimensionIri { <https://environment.ld.admin.ch/foen/nfi/unitOfReference> }
      ?observation ?dimensionIri ?versionedValue .
      ?versionedValue schema:sameAs ?unversionedValue .
    }
  }
} UNION {
  {
    SELECT DISTINCT ?dimensionIri ?versionedValue ?unversionedValue WHERE {
      { #pragma evaluate on
        SELECT ?observation WHERE {
          VALUES ?dimensionIri { <https://environment.ld.admin.ch/foen/nfi/unitOfReference> }
              <https://environment.ld.admin.ch/foen/nfi/nfi_T-changes/cube/2024-1> cube:observationConstraint/sh:property ?dimension .
              ?dimension sh:path ?dimensionIri .
              FILTER NOT EXISTS { ?dimension schema:version ?version . }
              FILTER NOT EXISTS { ?dimension sh:in ?in . }
          <https://environment.ld.admin.ch/foen/nfi/nfi_T-changes/cube/2024-1> cube:observationSet/cube:observation ?observation .

        }
      }
      VALUES ?dimensionIri { <https://environment.ld.admin.ch/foen/nfi/unitOfReference> }
      ?observation ?dimensionIri ?versionedValue .
      BIND(?versionedValue as ?unversionedValue)
    }
  }
}

OPTIONAL { ?versionedValue schema:name ?name_de . FILTER(LANG(?name_de) = "de") } OPTIONAL { ?versionedValue schema:name ?name_fr . FILTER(LANG(?name_fr) = "fr") } OPTIONAL { ?versionedValue schema:name ?name_it . FILTER(LANG(?name_it) = "it") } OPTIONAL { ?versionedValue schema:name ?name_en . FILTER(LANG(?nameen) = "en") } OPTIONAL { ?versionedValue schema:name ?name . FILTER(LANG(?name_) = "") } BIND(COALESCE(?name_de, ?name_fr, ?name_it, ?nameen, ?name) AS ?name) OPTIONAL { ?versionedValue schema:description ?description_de . FILTER(LANG(?description_de) = "de") } OPTIONAL { ?versionedValue schema:description ?description_fr . FILTER(LANG(?description_fr) = "fr") } OPTIONAL { ?versionedValue schema:description ?description_it . FILTER(LANG(?description_it) = "it") } OPTIONAL { ?versionedValue schema:description ?description_en . FILTER(LANG(?descriptionen) = "en") } OPTIONAL { ?versionedValue schema:description ?description . FILTER(LANG(?description_) = "") } BIND(COALESCE(?description_de, ?description_fr, ?description_it, ?descriptionen, ?description) AS ?description) OPTIONAL { ?versionedValue schema:alternateName ?alternateName_de . FILTER(LANG(?alternateName_de) = "de") } OPTIONAL { ?versionedValue schema:alternateName ?alternateName_fr . FILTER(LANG(?alternateName_fr) = "fr") } OPTIONAL { ?versionedValue schema:alternateName ?alternateName_it . FILTER(LANG(?alternateName_it) = "it") } OPTIONAL { ?versionedValue schema:alternateName ?alternateName_en . FILTER(LANG(?alternateNameen) = "en") } OPTIONAL { ?versionedValue schema:alternateName ?alternateName . FILTER(LANG(?alternateName_) = "") } BIND(COALESCE(?alternateName_de, ?alternateName_fr, ?alternateName_it, ?alternateNameen, ?alternateName) AS ?alternateName) OPTIONAL { ?versionedValue schema:identifier ?identifier . } OPTIONAL { ?versionedValue schema:position ?position . } OPTIONAL { ?versionedValue schema:color ?color . } OPTIONAL { ?versionedValue geo:hasGeometry ?geometry . } OPTIONAL { ?versionedValue schema:latitude ?latitude . } OPTIONAL { ?versionedValue schema:longitude ?longitude . } }


- query 2
```sparql
#pragma describe.strategy cbd
#pragma join.hash off

DESCRIBE ?subject WHERE {
  {
    BIND(<https://environment.ld.admin.ch/foen/gefahren-waldbrand-warnung/1> AS ?subject)
  }
  UNION
  {
    ?subject <http://schema.org/hasPart> <https://environment.ld.admin.ch/foen/gefahren-waldbrand-warnung/1> .
  }
}

DESCRIBE https://environment.ld.admin.ch/foen/gefahren-waldbrand-warnung/1/shape/

bprusinowski commented 3 months ago

Hi @Rdataflow, thanks again for catching this 💯 I am off for the end of this week and will tackle this once I am back on Monday ⛰️

Rdataflow commented 3 months ago

@adintegra please coordinate this fix makes it into the upcoming release