attic-labs / noms

The versioned, forkable, syncable database
Apache License 2.0
7.45k stars 267 forks source link

use refWalker for Value.WalkRefs #3756

Closed ghost closed 6 years ago

ghost commented 6 years ago

@cmasone-attic this resulted in about 13% improvement in csv-import. From looking at the profile, the main issue was that String.WalkRefs was calling WalkValues, which was allocating strings as it decoded them. using the refWalker avoid this.

I'm not totally sure why this causes tests to fail. Can you investigate?

Also, FWIW, after this patch, if I do the experiment of short-circuiting the grandchild strategy and eagerly writing values, it has no noticeable affect, so maybe the other stuff about refactoring layering of snappy encode/decode isn't high value right now.