search

3-Round-Stones / callimachus

Callimachus is a highly scalable platform for creating and running data-driven websites
Other
95 stars 24 forks source link

Potential dangers of reverse lookup in "describe" view #183

Closed edwardsph closed 8 years ago

edwardsph commented 9 years ago

Whilst viewing https://usepa.3roundstones.net/usepa/data/rcra?query=describe+%3Chttp://usepa.3roundstones.net/rcra/handlers/OHD000816801%3E I made the mistake of clicking the chevrons against the object http://usepa.3roundstones.net/rcra/land-type/P (the landType predicate appears after the list of hasActivity predicates). I got no response and the server needed a restart after this due to running out of memory. As soon as I clicked it I realised what would happen but it was too late. I believe the problem is that the describe page, as well as describing the object specified, includes a reverse lookup of triples referencing the object. In this case there would have been millions so it highlights how easy it is to cause a problem when you don't have a good understanding of the data. I was clicking around the describe page to try and gain this understanding.

Does the describe page use a select query to perform that reverse lookup and is there a limit on it? Is there anything we can do to protect users from mistakes like this?

prototypo commented 9 years ago

Thanks, Pete!

Note that this should be addressed as part of Issue #182 in Callimachus 1.5.

catch-point commented 9 years ago

This should be raised as an issue in Sesame's DESCRIBE query implementation.

catch-point commented 9 years ago

See also https://openrdf.atlassian.net/browse/SES-1904

prototypo commented 9 years ago

Resolution by phone: Perform a count of the describe on each URI in the query results when building the page. The hyperlink on the chevrons will resolve to the SPARQL query page populated with the generated query in the event that the count is "too high", and will include an error message saying that the number of results is too large. Small counts will result in the existing behavior.