search

tomeichlersmith / hdtree

columnar, ragged data with a dynamic, runtime-defined schema
https://tomeichlersmith.github.io/hdtree/
2 stars 1 forks source link

draft Python API #11

Open tomeichlersmith opened 1 year ago

tomeichlersmith commented 1 year ago

Similar to the C++ API, there already exists a lower-level HDF5 API that will do the heavy lifting for us. h5py

My general plan would be to make the interface uproot-like where browsing can be done without loading data into memory and then users can request certain trees/branches/groups of them to be loaded into a specific object in memory (dataframe, numpy, awkward, etc..)

tomeichlersmith commented 12 months ago

Another project already doing something similar - https://github.com/mattbellis/hepfile Not sure how close hepfile's HDF5 schema is to HDTree's but I can play around with it and see.