search

attic-labs / noms

The versioned, forkable, syncable database
Apache License 2.0
7.45k stars 266 forks source link

request set & list elements in batch #3660

Closed phritz closed 7 years ago

phritz commented 7 years ago

Load all leaf nodes for sets and lists before iterating them. Previously it loaded them on demand while iterating, meaning access to the underlying chunks was serialized.

@rafael-atticlabs there are some questions in comments for you

This the first part of what's required for toward https://github.com/attic-labs/noms/issues/3619

ghost commented 7 years ago

Well done, @phritz. In general, this looks good. I'm going to defer review to @arv, since this is his code area, and let him answer most of the questions.

About tree level, there are tests in types that keep adding elements until col.sequence().treeLevel() > 0, but that's not public API and available from this package. One route would be to add some public API. Lacking public API to directly return the tree level, you can also tell when a collection is more than one level deep by calling WalkRefs (which will start visiting leaf-nodes as soon as it's more than one level).

phritz commented 7 years ago

About tree level, there are tests in types that keep adding elements until col.sequence().treeLevel() > 0, but that's not public API and available from this package. One route would be to add some public API

That's fine I don't need an API, was just wondering if there was a different way than brute force.