search

UCBerkeleySETI / rawspec

6 stars 7 forks source link

Support for 16 and 32 bit input #41

Open wfarah opened 2 years ago

wfarah commented 2 years ago

rawspec currently supports an 8bit and more recently a 4bit input. The ATA beamformer will output samples at 16 and 32 bit (half and single precision float). The beamformer output will be saved in a guppi format, which I currently assume can support this bit rate. It would be a great addition to rawspec to make it handle the above input bit size. It's also common in astronomy to use 16 and 32 bit integers, so maybe this is worth a discussion as to how this could be treated within the code. Maybe an additional guppi header parameter

texadactyl commented 2 years ago

All of my GBT testing has been with 32-bit floats. So, this works. I do not have access to a .raw file with 16-bit floats. @david-macmahon : tried 16-bit floats yet?

Integer element formats? That's another question for @david-macmahon .

The FBH5 output code doesn't care what its writing.

wfarah commented 2 years ago

@texadactyl I can produce a half-precision file, but the values there will kinda be non-sense. At least until I can calibrate the beamformer properly

david-macmahon commented 2 years ago

rawspec currently supports 8+8 bit complex integer data and 16+16 bit integer data. The GUPPI RAW format has historically been limited to integer data. There is no "datatype" header field defined to indicate what type of values are in the data blocks other than NBITS which specifies how many bits each real/imaginary component has. We could invent a new header field to indicate integer or float, but other GUPPI RAW readers (e.g. dspsr) would ignore it until they get modified to accommodate it. I'm not sure it's really worth all the effort though. Why not just re-quantize to 8 or 16 bit integers before writing out the coherent beams? The data are coming in as 8 or 16 bit values anyway, would the dynamic range would increase so much that we require floating point output?

texadactyl commented 2 years ago

@wfarah Have you reached consensus with @david-macmahon ? Does this issue need a redefinition or ?

siemion commented 2 years ago

nbits=32 is widely recognized implicitly in pulsar land as meaning floating point. I think NBITS = 32 is a reasonable hack at this point

radonnachie commented 2 years ago

When it comes to how to trigger the integer_data flag, I believe we need primarily to decide how we want other programs to react to floating point data: do we want them to 🎉 break/error out/scream, or do we want them to 🚀 merely continue ignorantly. Negative NBITS values as a flag should fall into intention 🎉 . A new FITS key-value such as DATATYPE!='INTEGER' should fall into intention 🚀 .

Could we tally votes with reactions?