davidclemens / Dingi

Dingi is a toolbox for processing data collected by marine research gear.
MIT License
1 stars 0 forks source link

BIGO conductivity file import bug #138

Closed davidclemens closed 1 year ago

davidclemens commented 1 year ago

System information

-----------------------------------------------------------------------------------------------------
MATLAB Version: 9.3.0.713579 (R2017b)
MATLAB License Number: STUDENT
Operating System: Mac OS X  Version: 10.15.7 Build: 19H2 
Java Version: Java 1.8.0_121-b13 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
-----------------------------------------------------------------------------------------------------
MATLAB                                                Version 9.3         (R2017b)    
Simulink                                              Version 9.0         (R2017b)    
Aerospace Toolbox                                     Version 2.20        (R2017b)    
Control System Toolbox                                Version 10.3        (R2017b)    
Curve Fitting Toolbox                                 Version 3.5.6       (R2017b)    
DSP System Toolbox                                    Version 9.5         (R2017b)    
Dingi                                                 Version 0.1.0b1     (2022.5.0b1)
Image Processing Toolbox                              Version 10.1        (R2017b)    
Instrument Control Toolbox                            Version 3.12        (R2017b)    
Mapping Toolbox                                       Version 4.5.1       (R2017b)    
Optimization Toolbox                                  Version 8.0         (R2017b)    
Signal Processing Toolbox                             Version 7.5         (R2017b)    
Simulink Control Design                               Version 5.0         (R2017b)    
Statistics and Machine Learning Toolbox               Version 11.2        (R2017b)    
Symbolic Math Toolbox                                 Version 8.0         (R2017b)    
Utilities                                             Version 1.5.1       (2022.11.1) 

Bug description

What should have happened?

Nominal import of a BIGO conductivity file (CONDUCT.TXT). What did actually happened?

The import does not error but causes artefacts in the imported data with alternating rows of data patterns.

Steps to reproduce the bug

Import a AL570 BIGO conductivity file with DataKit.dataPool.readBigoConductivityCell (Note that this is a protected method). Do the same for an EMB238 BIGO conductivity file.

Causes & solutions

The conductivity files often have a mismatch between the number of header columns and data columns. Therefore, the formatSpec for textscan does not match the data stream, causing errors (if the formatSpec has more specifiers than the file has data) or shifts in data (if the formatSpec does not have enough specifiers).

Make the import process more robust to handle these errors gracefully.