digitalbazaar / rdf-canonize

An implementation of the RDF Dataset Normalization Algorithm in JavaScript.
Other
22 stars 13 forks source link

Async version of urdna2015 does not properly batch process hashing in parallel #54

Open dlongley opened 1 year ago

dlongley commented 1 year ago

The async version of URDNA2015 doesn't take full advantage of multiple processes / thread pooling to process hashes. It was converted some time ago from non-promises to promises and this was probably lost in translation somewhere. When hashing 1st degree quads, for example, each hash call is awaited vs. batching the returned promise and awaiting up to N-many ops at once. This should be fixed to improve performance / make better use of available resources.