vaexio / vaex

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀
https://vaex.io
MIT License
8.27k stars 590 forks source link

[BUG-REPORT] #1443

Open fredifqh opened 3 years ago

fredifqh commented 3 years ago

Dear Marteen,

I can't open a fits file (attached in the message) using:

import vaex
vaex.open('test.fits')

I get the following error

ERROR:MainThread:vaex:error opening 'test.fits'

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-34-c09b06bf0807> in <module>
----> 1 vaex.open('test.fits')

~/anaconda3/lib/python3.8/site-packages/vaex/__init__.py in open(path, convert, shuffle, fs_options, fs, *args, **kwargs)
    221                 ds = vaex.dataset.open(path_output, fs_options=fs_options, fs=fs, **kwargs)
    222             else:
--> 223                 ds = vaex.dataset.open(path, fs_options=fs_options, fs=fs, **kwargs)
    224             df = vaex.from_dataset(ds)
    225             if df is None:

~/anaconda3/lib/python3.8/site-packages/vaex/dataset.py in open(path, fs_options, fs, *args, **kwargs)
     80     failures = "\n".join([f'\n-----{who}-----\n:' + vaex.utils.format_exception_trace(e) for e, who in failures])
     81     if failures:
---> 82         raise IOError(f'Cannot open {path}, failures: {failures}.')
     83     else:
     84         raise IOError(f'Cannot open {path} nobody knows how to read it.')

OSError: Cannot open test.fits, failures: 
-----<class 'vaex.astro.fits.FitsBinTable'>-----
:Traceback (most recent call last):
  File "/home/fredi/anaconda3/lib/python3.8/site-packages/vaex/dataset.py", line 76, in open
    return opener.open(path, fs_options=fs_options, fs=fs, *args, **kwargs)
  File "/home/fredi/anaconda3/lib/python3.8/site-packages/vaex/dataset.py", line 1447, in open
    return cls(path, *args, **kwargs)
  File "/home/fredi/anaconda3/lib/python3.8/site-packages/vaex/astro/fits.py", line 91, in __init__
    self._try_votable(fitsfile[0])
  File "/home/fredi/anaconda3/lib/python3.8/site-packages/vaex/astro/fits.py", line 130, in _try_votable
    self.descriptions[clean_name] = field.description
AttributeError: 'FitsBinTable' object has no attribute 'descriptions'

Thanks for your answer in advance test.zip

matteobachetti commented 2 years ago

Hello, I'm having the same problem.

----> 1 fromfits = vaex.open("file.evt")

~/anaconda3/envs/py38/lib/python3.8/site-packages/vaex/__init__.py in open(path, convert, shuffle, fs_options, fs, *args, **kwargs)
    223                 ds = vaex.dataset.open(path_output, fs_options=fs_options, fs=fs, **kwargs)
    224             else:
--> 225                 ds = vaex.dataset.open(path, fs_options=fs_options, fs=fs, **kwargs)
    226             df = vaex.from_dataset(ds)
    227             if df is None:

~/anaconda3/envs/py38/lib/python3.8/site-packages/vaex/dataset.py in open(path, fs_options, fs, *args, **kwargs)
     85         raise IOError(f'Cannot open {path}, failures: {failures}.')
     86     else:
---> 87         raise IOError(f'Cannot open {path} nobody knows how to read it.')
     88 
     89 

OSError: Cannot open file.evt nobody knows how to read it.
JovanVeljanoski commented 2 years ago

@matteobachetti

I don't know what kind of file that .evt is, but in any case it is not supported by vaex. Please use a memory mappable file format like hdf5 (column based), arrow, parquet or csv.

matteobachetti commented 2 years ago

@JovanVeljanoski oh, sorry, my bad. It's an OGIP-compliant FITS file. evt is so commonly used in X-ray astronomy that I didn't think about the extension! Thanks for your time

JovanVeljanoski commented 2 years ago

Is this still an issue? Or did adjusting the extension did the trick?