search

vaexio / vaex

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀
https://vaex.io
MIT License
8.22k stars 590 forks source link

[BUG-REPORT] Failed to open hdf5 file stored on AWS S3 #2315

Open rongxinyu opened 1 year ago

rongxinyu commented 1 year ago

Description

  1. I created a dataframe and exported it to an hdf5 file with df.export_hdf5('test.hdf5')
  2. I can open the hdf5 file with call vaex.open('test.hdf5')
  3. I copied the file to aws with aws s3 cp test.hdf5 s3://<bucket>/playground/test.hdf5
  4. I got the following error when running vaex.open('s3://<bucket>/playground/test.hdf5'): AttributeError: 'ColumnArrowLazyCast' object has no attribute 'buffers'

Software information