Closed jindrichmynarz closed 7 years ago
Scrollable cursors are implemented in LinkedPipes ETL components http://etl.linkedpipes.com/components/e-sparqlendpointselectscrollablecursor and http://etl.linkedpipes.com/components/e-sparqlendpointconstructscrollablecursor
All DPUs loading data via SPARQL queries using
ORDER BY
(such as XSLT DPU) should use Virtuoso's scrollable cursors (see Virtuoso's documentation, section "Example: Prevent Limits of Sorted LIMIT/OFFSET query"). WhenOFFSET
in an ordered SPARQL query exceeds Virtuoso's settingMaxSortedTopRows
fromvirtuoso.ini
(typically set to 10-20K rows), the query fails with error message like the following:Temporary workaround for this issue is to increase the
MaxSortedTopRows
setting, but the solution is to use a sub-SELECT withORDER BY
wrapped in SELECT query withOFFSET
andLIMIT
. For example, the XSLT DPU uses the query:This query with scrollable cursor that allows loading larger data could look like the following: