Open lucaswerkmeister opened 6 years ago
So I’ve spent some more time trying to arrive at shape expressions that don’t crash shex.js, and I found something odd. Given the data and schema in this gist, validating http://www.wikidata.org/entity/Q42 against http://www.wikidata.org/entity/Q5 will succeed – but only because I’ve commented out two wdt:P40
(“child”) links: with those lines not commented out, shex.js crashes. Is shex.js perhaps trying to validate the same nodes against the same shapes again and again, because of the circular references between child items (P40) and parent items (P22, P25)?
I tried reproducing this in shex-simple but I don't think I've got the right shapemap. I didn't see http://www.wikidata.org/entity/Q19296 in the gist data.
Sorry, the example in the gist is completely independent from the one in the original bug report. The minimal schema from the issue description (ex:A
, ex:B
) causes an error when you let shex.js download the data for Q19296, but not when you use the data file from the gist (probably because the gist doesn’t contain label or description triples). The data file from the gist, on the other hand, produces an error only with the schema from the gist, and only if the wdt:P40
lines are uncommented.
This is a lot easier to see if you use Chrome Latest and then profile the Performance while validating. Areas in the source code show highlighted function call timing in ms after profile.
Each validated object uses a certain amount of memory for the object plus the graph it retains in bytes, and how much depends on the Object Properties. (your mileage may vary), but take a look using Chrome Memory Allocation timeline and the Containment / Retained Size.
Tx, @thadguidry! I'd never ventured that far to the right in the Chrome debugger. I guess the issue is that I need t throw stuff away, in particular, error reports, once they're represented in the UI.
I’ve reduced my shape expression to this fairly short snippet:
Let’s say you’ve saved this as
/tmp/ex.shex
. If you then try to validate this against Q19296, a fairly small Wikidata example item, using the commandthen Node.js will fairly quickly (within less than ten seconds on my system) crash with a fatal out-of-memory error. (You probably don’t want to try this in a browser – for me, that hung up the whole system for a while.)
Commenting out any of the lines in the ShEx code makes the validation pass, so it feels like this is not a problem with any particular part of the shape, but rather like shex.js is just getting overwhelmed by the sheer amount of labels and descriptions?
Any ideas what could be done here? :/