search

alex-s-gardner / H5ToTable.jl

MIT License
1 stars 0 forks source link

Questions about implementation #1

Open alex-s-gardner opened 5 months ago

alex-s-gardner commented 5 months ago
  1. is the Tuple of Pairs{Symbol, H5var/H5att} an intuitive way to specify the variable names and locations?
  2. Is returning a NamedTuple sufficient or do we need to include the Tables.jl API?
evetion commented 2 months ago
  1. Symbol for sure. Not sure about merging H5var/H5att, you could also pass them seperately like h5table(h5; vars, attrs). Besides, I would code shortcuts, like Pair{Symbol, String}, so you can do latitude => "/gt1l/land_ice_segments/latitude". One could possibly even derive the name from a single string by taking the last of the split on /.
  2. A namedtuple should work out of the box right? Maybe test that it implements the column based iterator interface, and see whether DataFrame has to copy or not.
evetion commented 2 months ago

Nitpick, I would use step instead of stride to keep it similar to the built-in range we have.

alex-s-gardner commented 2 months ago

@evetion what additional functionality do you need in H5ToTable to satisfy SpaceLiDARs needs? Is it just the ability to pass a function to apply to variables upon read?

evetion commented 2 months ago

I brainstormed about it in https://github.com/evetion/SpaceLiDAR.jl/issues/58:

size/datatype and its shared dimensions(!), as well as a function that would be applied on loading

But it might be better suited for an extended or prepared var that implements the same interface? These requirements are for automatically including the dimensions of requested variables, being slightly more type stable, and being able to error when two incompatible variables (different lengths) are selected for the same table.