`trim_iq` doubles length of data array

[ecbland]

When creating an iqdat file using trim_iq, the data array has double the number of elements as the input file (and the output file approximately doubles in size).

I expect that this behaviour comes from the iq library rather than trim_iq itself. Does anyone know if this behaviour is intended?

// ORIGINAL IQDAT FILE
$ dmapdump 20180218.1401.00.bks.iqdat | grep '"data' | head
    short   "data" [54472]
    short   "data" [54472]
    short   "data" [54472]
    short   "data" [54472]
    short   "data" [54472]
    short   "data" [54472]
    short   "data" [54472]
    short   "data" [56948]
    short   "data" [54472]
    short   "data" [54472]

// OUTPUT FROM TRIM_IQ (no actual trimming)
$ trim_iq 20180218.1401.00.bks.iqdat | dmapdump | grep '"data' | head
    short   "data" [108944]
    short   "data" [108944]
    short   "data" [108944]
    short   "data" [108944]
    short   "data" [108944]
    short   "data" [108944]
    short   "data" [108944]
    short   "data" [113896]
    short   "data" [108944]
    short   "data" [108944]

[egthomas]

@ecbland I think this has to do with a change in how IQ samples were treated in the QNX4 vs Linux/QNX6 ROS. The short answer can be found in the IQEncode function:

https://github.com/SuperDARN/rst/blob/734542037d86b24475d8da571ed668659a858e27/codebase/superdarn/src.lib/tk/iq.1.7/src/iqwrite.c#L64

From what I can tell, the QNX4 code set an nchannel parameter as either one or two depending on whether XCFs were present. This nchannel was used to both populate the IQ data array, and then set the value of an intermediate rxchn and then iq->chnnum which is stored in the iqdat files themselves:

https://github.com/SuperDARN/ros-archive/blob/a87c608b5e96d33fc1267fa3a31d170aa5d4aa28/qnx/code/src.lib/radarqnx4/integrategc214.1.29/src/integrategc214.c#L230-L241

https://github.com/SuperDARN/ros-archive/blob/a87c608b5e96d33fc1267fa3a31d170aa5d4aa28/qnx/usr/code/src.lib/radarqnx4/site/site.bks.2.00/src/hdw.c#L313-L323

https://github.com/SuperDARN/ros-archive/blob/a87c608b5e96d33fc1267fa3a31d170aa5d4aa28/qnx/usr/code/src.lib/radarqnx4/ops.1.17/src/build.c#L160

On the other hand, in the Linux/QNX6 ROS on which the RST 3+ series of code is based, it appears rxchn is instead treated as the number of receiver channels (because the MSI-style QNX6 radars were theoretically capable of up to 4 independent transmit/receive channels):

https://github.com/SuperDARN/rst-archive/blob/6b3a8d205d008630f34ae08cafc72deec4234562/usr/codebase/superdarn/src.lib/os/site.cve.1.3/src/site.c#L118-L120

For most of the MSI-style QNX6 radars rxchn (and thus iq->chnnum) is likely always set to one (eg Fort Hays, Christmas Valley, Iceland), so maybe that extra multiplication by 2 in IQEncode makes sense. But when applied to iqdat files from any other radar that probably won't be correct.

[egthomas]

after a little further digging, it seems the way the size of the IQ sample array is calculated in IQEncode changed between versions 3.2 and 3.3 of the RST

[egthomas]

I think the appropriate way to handle this is to remove the extra multiplication by 2 in IQEncode, modify the ROS code used by the Linux/QNX6-style radars to set iq->chnnum equal to 2, and make the corresponding edits to make_raw . That should bring the iqdat files of as many of the ROS-style radars in line as possible.

[ecbland]

Fixed with #566, thanks @egthomas 😄