UCBerkeleySETI / blimpy

Breakthrough Listen I/O Methods for Python
https://blimpy.readthedocs.io
BSD 3-Clause "New" or "Revised" License
46 stars 40 forks source link

Bug with h52fil #41

Closed mattlebofsky closed 6 years ago

mattlebofsky commented 6 years ago

Trying to convert a filterbank file to hdf5 at GB using blimpy 1.2.1 (though I tried 1.2.0 and 1.1.8). And I get this error:

h52fil /mnt_bls4/datax3/holding/spliced_blc0001020304050607_guppi_57708_17466_HIP19335_0003.gpuspec.0002.h5 -o /datax3/users/sci -n filfromf5.fil Traceback (most recent call last): File "/opt/pyve/eepy/bin/h52fil", line 11, in load_entry_point('blimpy==1.2.1', 'console_scripts', 'h52fil')() File "/opt/pyve/eepy/local/lib/python2.7/site-packages/blimpy-1.2.1-py2.7.egg/blimpy/h52fil.py", line 60, in cmd_tool make_fil_file(filename, out_dir = opts.out_dir, new_filename=opts.new_filename) File "/opt/pyve/eepy/local/lib/python2.7/site-packages/blimpy-1.2.1-py2.7.egg/blimpy/h52fil.py", line 41, in make_fil_file fil_file.write_to_fil(new_filename) File "/opt/pyve/eepy/local/lib/python2.7/site-packages/blimpy-1.2.1-py2.7.egg/blimpy/waterfall.py", line 227, in write_to_fil self.write_to_fil_light(filename_out) File "/opt/pyve/eepy/local/lib/python2.7/site-packages/blimpy-1.2.1-py2.7.egg/blimpy/waterfall.py", line 274, in write_to_fil_light fileh.write(generate_sigproc_header(self)) #generate_sigproc_header comes from sigproc.py File "/opt/pyve/eepy/local/lib/python2.7/site-packages/blimpy-1.2.1-py2.7.egg/blimpy/sigproc.py", line 375, in generate_sigproc_header header_string += to_sigproc_keyword(keyword, f.header[keyword]) File "/opt/pyve/eepy/local/lib/python2.7/site-packages/blimpy-1.2.1-py2.7.egg/blimpy/sigproc.py", line 350, in to_sigproc_keyword return np.int32(len(keyword)).tostring() + keyword + value_dtype(value).tostring() UnicodeDecodeError: 'ascii' codec can't decode byte 0xaa in position 4: ordinal not in range(128)

Not sure how to proceed... I was told to create this issue...

telegraphic commented 6 years ago

Hi Matt, does the file open with the siggproc header tool? Can you please point your me toward the data file causing it, or is it all filterbanks?

On Thu., 12 Jul. 2018, 08:20 Matt Lebofsky - SETI, notifications@github.com wrote:

Assigned #41 https://github.com/UCBerkeleySETI/blimpy/issues/41 to @telegraphic https://github.com/telegraphic.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/UCBerkeleySETI/blimpy/issues/41#event-1729058243, or mute the thread https://github.com/notifications/unsubscribe-auth/AAriI4-719GqtpUMq2HQzC9YOS3R8Ziuks5uFnoQgaJpZM4VL7vb .

mattlebofsky commented 6 years ago

Seems to be all hdf5 files (I'm trying to convert h5 files back into filterbank). So the sigproc header tool isn't relevant. But watutil does work to display the header (i.e. watutil -i filename)

You can find a bunch of files to test in, say, bls5:/datax/holding

On Wed, Jul 11, 2018 at 4:22 PM, Danny Price notifications@github.com wrote:

Hi Matt, does the file open with the siggproc header tool? Can you please point your me toward the data file causing it, or is it all filterbanks?

On Thu., 12 Jul. 2018, 08:20 Matt Lebofsky - SETI, < notifications@github.com> wrote:

Assigned #41 https://github.com/UCBerkeleySETI/blimpy/issues/41 to @telegraphic https://github.com/telegraphic.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/UCBerkeleySETI/blimpy/issues/41#event-1729058243, or mute the thread https://github.com/notifications/unsubscribe-auth/AAriI4- 719GqtpUMq2HQzC9YOS3R8Ziuks5uFnoQgaJpZM4VL7vb .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/UCBerkeleySETI/blimpy/issues/41#issuecomment-404340621, or mute the thread https://github.com/notifications/unsubscribe-auth/AH2JCh9q544x-Kd9LiWvIN4aiMtlYhsxks5uFoi1gaJpZM4VL7vb .

-- Matt Lebofsky - Breakthrough Listen - lebofsky@berkeley.edu - (510) 207-0832

telegraphic commented 6 years ago

H'ok, I think this is fixed in my last set of changes. This was specifically related to Py2/Py3 compatibility (AGAIN). I also found a missing b in b'src_raj that was causing Py3 (and only Py3) to load the wrong Angle value to sigproc filterbank, converting hourangle into degrees. 🤕 🐍🐍🐍

jeenriquez commented 6 years ago

Thanks Danny! I'm looking at all the changes and merging soon. A conversation when you are here: what about stop supporting python2 by the end of the year? (But not earlier!!)