datreant / datreant.data

convenient data storage and retrieval in HDF5 for Treants
http://datreant.org/
BSD 3-Clause "New" or "Revised" License
1 stars 1 forks source link

pytable warns about possible performance issues with lots of columns for dataframes #17

Open kain88-de opened 7 years ago

kain88-de commented 7 years ago

When I store pandas objects with a LOT of columns I get the following error message. I haven't yet noticed any problems with memory though. I assume we don't need to care about this error since we always load the complete hdf5 file into memory and don't to any mmap tricks on the hdf5 file.

/home/max/conda/lib/python2.7/site-packages/tables/table.py:1039: PerformanceWarning: table ``/main/table`` is exceeding the recommended maximum number of columns (512); be ready to see PyTables asking for *lots* of memory and possibly slow I/O
  PerformanceWarning)

Should we just silence this warning as well?