ConnectedDataConstructor gives num triples < limit despite truncating

[weisenje]

Summary: SparqlToXLibUtil.generateConstructConnected() was intended to guarantee that if data is truncated, then number of triples returned will be >= limit. Consuming processes count on this to know whether the data may be truncated. However, we are seeing data truncated and the number of triples below the limit.

Details/Example: In SPARQLgraph, node expansion uses ADD_TRIPLES_MAX = 1000. This number appears as the “limit” argument to generateConstructConnected(), which incorrectly assumes 3 triples per line and thus generates SPARQL with LIMIT 334, which truncates the data. In the case observed, we get 2 triples per line (the connected instance type and connecting predicate), for a total of 667 triples. Since 667 is under the max of 1000, SPARQLgraph does not give the "using a subset" message.

Note: For connected literals (vs URIs), may only see 1 triple per line. Note: The “3 triples per line” may have come from a use case where there was a predicate in extraPredicatesList.

[weisenje]

To reproduce:

• Load de-identified RACK data
• Construct Activity uri://semtk#72af4c65-de59-4bf4-983a-d96757da70b9
• Double-click the Activity to expand it
• This SPARQL query is generated:

  CONSTRUCT { ?s ?p ?o.  ?s a ?st. ?o a ?ot. ?s ?xp ?xs. ?o ?xp ?xo. } 
  WHERE { {  
  BIND ( <uri://semtk#72af4c65-de59-4bf4-983a-d96757da70b9>  as ?s) . ?s ?p ?o .
  OPTIONAL { ?o a ?ot } .  
  } UNION {
  BIND ( <uri://semtk#72af4c65-de59-4bf4-983a-d96757da70b9>  as ?o) . ?s ?p ?o .
  OPTIONAL { ?s a ?st } .  
  } } LIMIT 334

• Generates ~2 triples per “line”, for 667 total triples:

  <uri://semtk#72af4c65-de59-4bf4-983a-d96757da70b9> http://arcos.rack/PROV-S#identifier "bd721f8e-3b9d-447d-bdf2-117d12afe494" 
  <uri://semtk#72af4c65-de59-4bf4-983a-d96757da70b9> http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://arcos.rack/PROV-S#ACTIVITY
  And many of these pairs:
  <uri://semtk#7039d111-2de2-4967-8fa7-0e9ab78aa9c6> http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://arcos.ah-64d/Boeing#SRS_Req .
  <uri://semtk#7039d111-2de2-4967-8fa7-0e9ab78aa9c6> http://arcos.rack/PROV-S#dataInsertedBy <uri://semtk#72af4c65-de59-4bf4-983a-d96757da70b9> .
  <uri://semtk#5083eafd-1413-499c-af83-84426fda79a1> http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://arcos.ah-64d/Boeing#SRS_Req .
  <uri://semtk#5083eafd-1413-499c-af83-84426fda79a1> http://arcos.rack/PROV-S#dataInsertedBy <uri://semtk#72af4c65-de59-4bf4-983a-d96757da70b9> .
  <uri://semtk#e5db401a-7e87-4799-9b2c-57ffed89ff70> http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://arcos.ah-64d/Boeing#SubDD_Req.
  <uri://semtk#e5db401a-7e87-4799-9b2c-57ffed89ff70> http://arcos.rack/PROV-S#dataInsertedBy <uri://semtk#72af4c65-de59-4bf4-983a-d96757da70b9>

  Results count 667 < 1000 => SPARQLgraph gives “warning” message (“this could overload your browser”) instead of the “using a subset” message