uchicago-cs / deepdish

Flexible HDF5 saving/loading and other data science tools from the University of Chicago
http://deepdish.io
BSD 3-Clause "New" or "Revised" License
270 stars 59 forks source link

Elegent way to combine multiple h5 files into a single h5 file #33

Closed aseembits93 closed 4 years ago

aseembits93 commented 6 years ago

Hi! Thank you for making this tool available for researchers. It is extremely helpful for data scientists and deep learning researchers. So, my problem is that I have multiple h5 files with the data as a dictionary with integer keys (0,1,...). I just want to combine these into a single file with integer keys. for example d1 = {0:['abc'],1:['def']}, d2={0:['qwe'],1:['rty']} I want d3 = {0:['abc'],1:['def'],2:['qwe'],3:['rty']} is there any way to do this without loading the dictionaries into RAM and writing a script to change the keys of the dictionaries? Thank you!