Channel ordering

[debortoli]

Due to the ordering of data channels, when I ran this code it broke reading my XTF file. I am only trying to access the port and starboard sonar data so changing line 228 in xtf_ctypes from

chan_info = [self.ChanInfo[i] for i in range(0, self.channel_count())] to chan_info = [self.ChanInfo[i] for i in range(0, 6) if self.ChanInfo[i].TypeOfChannel in [1,2]]

ended up fixing the problem.

I think this line change the makes the code more generalizable. Instead of assuming the relevant information is in the first two channels (as this code does if self.channel_count=2), it searches over all of the channels until it finds the two with port and starboard imagery.

[oysstu]

The code does not assume that the sonar channel data exists in the first two channels. The function channel_count computes the total number of channels based on the NumberOfXXXChannels fields in the file header, which determines the number of ChanInfo present. I'm hesitant to deviate from this logic, as any test for unused channels would rely on the chan_info being zero-initialized (which may not be true).

From this I suspect that the problem is one of the following

1. Your XTF file declares too few channels in the file header
2. Subbottom profiling sonar is currently included in the sonar_chan, which may cause problems. I don't have any XTF files with SBP available. If your file has this channel, there is a real possibility of something breaking.

On another note, this is from the XTF spec with respect to the CHANINFO fields

The CHANINFO structures for all sidescan channels will always precede the structures for the bathymetry channels.

It says nothing about the subbottom profiler, however.

If you could provide me with some more details regarding your XTF file and the channels, I will try to implement a fix when I understand the root cause.

[debortoli]

Yeah so my first two channels appear to be TypeOfChannel=0 (or subbuttom).

So when my XTF file is processed to this point, self.ChanInfo has 6 elements whose TypeOfChannel are 0,0,2,1,0,0 respectively. It doesn't appear as though there is any bathymetry data. So originally what would happen is my header would be read in and self.channel_count would equal 2, which makes sense, but since my first two elements are not port or stbd data, the reader would get stuck on:

# Read the data and output as a numpy array of the specified bytes-per-sample samples = buffer.read(n_samples * file_header.sonar_info[i].BytesPerSample) if not samples:
raise RuntimeError('File ended while reading data packets (file corrupt?)')

because file_header.sonar_info[i] would have value 0 instead of 2, I think because file_header.sonar_info did not have any data because the reader never came across elements with TypeOfChannel=1 or 2, because my first two elements were both of type 0.

While chan_info may not be zero based, could you just loop over all of them? For some reason I seem to remember reading that there were 6 channels? So that would limit things.

Thanks

[oysstu]

In that case, I suggest that I simply separate the sonar and subbottom channel info. I've pushed it to the repository. Could you try the current version to see if it works?

Out of curiosity, which HeaderType is used for the subbottom profiling XTFPingHeaders?

[debortoli]

Ok yeah we're getting closer. The problem for me (still remaining), is line 225 in xtf_ctypes: chan_info = [self.ChanInfo[i] for i in range(0, self.channel_count())] # type: List[XTFChanInfo]

because self.channel_count() equals 2 (in all of the XTF files I have seen) not all of the elements in self.ChanInfo are searched, only the first two are. Because the first two elements for me are TypeOfChannel=0 instead of TypeOfChannel=1 or 2, the updated code still does not work for me. The below code works for me, because its searching over all of self.ChanInfo:

chan_info = [self.ChanInfo[i] for i in range(0, self.channel_count())] # type: List[XTFChanInfo] self.subbottom_info = [x for x in chan_info if x.TypeOfChannel == XTFChannelType.subbottom] sonar_types = (XTFChannelType.port.value, XTFChannelType.stbd.value) self.sonar_info = [x for x in self.ChanInfo if x.TypeOfChannel in sonar_types] self.bathy_info = [x for x in self.ChanInfo if x.TypeOfChannel == XTFChannelType.bathy.value]

Theoretically I think you could get rid of the first two lines.

Still chasing down that HeaderType...

[oysstu]

In my opinion this seems like a flaw in the XTF format, perhaps there should've been a NumSubbottomChannels field in the header, or even better just a NumChannels. Perhaps NumEchoStrengthChannels should have been used for the subbottom profiler count. After all, if more than 6 channels is needed, the structure keeps on growing. There should be a count field indicating this, which also should include the subbottom channels.

To work around this I could implement your fix, but discarding elements that exceeds the sonar channel count just in case. self.sonar_info = [x for x in self.ChanInfo if x.TypeOfChannel in sonar_types][:self.NumberOfSonarChannels]

Could you also check what the SubChannelNumber in the ping header is set to for the sonar pings? I've been curious if this could be used to index into the ChanInfo structures, or if its related to the PingChanHeader.

[oysstu]

I've pushed the changes mentioned in my previous post, we'll see if it breaks something somewhere along the line. Please let me know how it works for your file.

[debortoli]

Ok sounds good thanks. I'm away for a bit but will check on it when I get back

[debortoli]

The latest change worked for my file and did not break on other files that worked with the previous code, so it should be good! Thanks.

The SubChannnelNumber is 0 for all types of sonar data (it does not matter port or starboard). This seems to be the case across files (for ones that worked previously and ones that now work).