tomeichlersmith / hdtree

columnar, ragged data with a dynamic, runtime-defined schema
https://tomeichlersmith.github.io/hdtree/
2 stars 1 forks source link

move Buffer into Branch for thread safety #5

Closed tomeichlersmith closed 1 year ago

tomeichlersmith commented 1 year ago

Goal: Different Branches can exist on different threads without issue (as long as one isn't a sub-branch of another).

tomeichlersmith commented 1 year ago

I'm thinking of a re-design that would make Reader/Writer more of a configuration class that simply holds some helper functions for interacting with H5 files. The general idea is to have the Branch object be something that holds a value and can be attached to an input file for reading or an output file for writing.

The actual buffers will only be held by atomic branches while parent branches would just call child branch methods after getting their attributes.

class Branch {
  // drop any reading buffers, attach us and sub-branches to new output file
  void attach(Reader& input);
  // load value of Branch from input (i.e. read value from buffer read from disk)
  void load();
  // flush any writing buffers, attach us and sub-branches to new output file
  // similar to function called `structure` right now
  void attach(Writer& output);
  // save current value of Branch into output (i.e. put value into buffer destined for disk)
  void save();
};
tomeichlersmith commented 1 year ago

this redesign has been done in https://github.com/tomeichlersmith/hdtree/commit/ed94934766afa7a5871de07bde8d826de9fff1d7

more documentation and testing needs to be done, but this is resolved