OSOceanAcoustics / echodataflow

Orchestrated sonar data processing workflow
https://echodataflow.readthedocs.io/en/latest/
MIT License
4 stars 1 forks source link

`echoshader_flow.py` flow does not assemble the correct MVBS dataset #124

Open leewujung opened 3 weeks ago

leewujung commented 3 weeks ago

One of the tasks of echoshader_flow.py is to assemble an MVBS dataset whose path is cached and accessed by the Panel app.

Currently it seems that there is some problem with grabbing the correct individual MVBS zarr files in assembling this combined MVBS dataset. As a results, even though the current datastore_shimada.yaml request for a 3 hour window, the actual MVBS dataset assembled in echodataflow/code/echodataflow/echodataflow-output/echoshader_flow/shimada/eshader.zarr only contains much less data, ~2 hrs 20 mins long only (see image below). I have checked the content of this eshader.zarr that it indeed contains only ~2 hrs 20 mins data.

Below is what is currently shown on the Panel app:

Screenshot 2024-08-17 at 1 29 37 PM

Below is what's in datastore_shimada.yaml

Screenshot 2024-08-17 at 1 34 39 PM