The ad2cp binary reader is too slow, which is blocking the processing of big adcp data files. For Example, SAIMOS facility cannot feasibly import a big file (~4gb).
The problem appears on the logic of the reader since it's sequentially detecting lengths/sizes even though those are pre-defined. The solution appears a vectorization/bulk read of the fields instead of using a long iterative loop.
Requirements:
[ ] #795
[ ] Change iterative block reading.
[ ] Test performance and correctness based on the previous reader.
The ad2cp binary reader is too slow, which is blocking the processing of big adcp data files. For Example, SAIMOS facility cannot feasibly import a big file (~4gb).
The problem appears on the logic of the reader since it's sequentially detecting lengths/sizes even though those are pre-defined. The solution appears a vectorization/bulk read of the fields instead of using a long iterative loop.
Requirements:
- [ ] Alternative/Extra: Create an OceanContour netcdf Parsersee https://github.com/aodn/imos-toolbox/issues/691