search

UCBerkeleySETI / blimpy

Breakthrough Listen I/O Methods for Python
https://blimpy.readthedocs.io
BSD 3-Clause "New" or "Revised" License
46 stars 40 forks source link

Create a 2-bit to 8-bit utility per @telegraphic #55

Open shamichatterjee opened 6 years ago

shamichatterjee commented 6 years ago

f = Waterfall('ParkesHTRU.fil', t_start=7800000, t_stop=7850000) f.info() -> Returns expected values from the header. f.plot_waterfall() -> May or may not be doing the right thing, but at leas the axis labels look right. But f.plot_spectrum() /opt/anaconda/lib/python3.6/site-packages/blimpy/filterbank.py in plot_spectrum(self, t, f_start, f_stop, logged, if_id, c, **kwargs) 546 plot_data = plot_data.mean() 547 else: --> 548 raise RuntimeError("Unknown integration %s" % t) 549 550 # Rebin to max number of points RuntimeError: Unknown integration all

I wonder if this is actually an error in unpacking the 2-bit data? If I try to read the data values I see values like -4.6255555765569165e+303 6.807427090242793e+301 -1.570574712372553e+108 but plot_waterfall() seems to be working around it. Or is it? Is there a way to get access to the data array and directly imshow it, for example? When I do d = f.data[:,0,:].transpose() then d.shape is (1024, 50000), as expected (1024 channels, 50,000 time samples) but I get garbage when trying to imshow(d) because it looks like the 2-bit values aren't unpacked correctly.

Please let me know how to better document this error so that you can see the issue! Thanks.

jeenriquez commented 6 years ago

Hi Shami, thanks for the report, could I ask which version of blimpy are you using?

Danny Price (@telegraphic) since you implemented the 2-bit handling I'm assigning this to you :)

telegraphic commented 6 years ago

Hey @shamichatterjee, does it work if you load the whole file? Can you point me toward a small 2-bit file with something interesting in it at a known location so I can work on it?

For 2-bit data you may be better off using sigpyproc (for now at least) - https://github.com/ewanbarr/sigpyproc

shamichatterjee commented 6 years ago

@telegraphic sorry, I used a file from the Parkes HTRU archive and don't really have anything smaller, but Vishal G should be able to provide one. It sure looks like the 2-bit values are still packed and the combined value is interpreted as a (random) float when I try to access the array.

I installed the current version via downloading a zip file from GitHub - setup.py lists version = '1.3.4'

I did try sigpyproc just now - it installed fine but trying to import led to a syntax error on a fairly innocuous line: raise ValueError,"readsamps must be > skipback value") I'll try to look into that but I suspect that's maybe a mess.

telegraphic commented 5 years ago

Labelling both a bug and an enhancement!

telegraphic commented 5 years ago

This relates to #25, which I will close as a duplicate issue

telegraphic commented 5 years ago

After some discussion, we think that a better approach is to create a 2-bit to 8-bit utility, and will only support uint8 and above.

A error should be raised if data are 2-bit, with direction in the docstring to the 2to8 utility.