uchicago-cs / deepdish

Flexible HDF5 saving/loading and other data science tools from the University of Chicago
http://deepdish.io
BSD 3-Clause "New" or "Revised" License
270 stars 59 forks source link

How to convert csv/txt file to hdf5? #24

Closed sundyCoder closed 7 years ago

sundyCoder commented 7 years ago

How to convert csv/txt file to hdf5?

asanakoy commented 7 years ago

For csv

import pandas as pd
df = pd.read_csv(filepath)
df.to_hdf(fout_filepath)
gustavla commented 7 years ago

I just want to add to @asanakoy's answer that deepdish is fully compatible with pandas too, essentially calling to_hdf internally. These two will create the same HDF5 file (both can be loaded with deepdish):

df.to_hdf(filename, key='table')
dd.io.save(filename, dict(table=df)

However, deepdish extends this by allowing you to save multiple data frames inside a single HDF5 file, intermixed with the other data types supported by deepdish:

dd.io.save(filename, dict(table=df, more_tables=[df1, df2, df3]))