Closed lsetiawan closed 6 months ago
Command to run the tests and capture the stdout:
python .ci_helpers/run-test.py --local --pytest-args="-vvrP" echopype/mask/api.py
pytest -vvrP echopype/tests/mask/test_mask.py::test_frequency_differencing
@lsetiawan : the code you pointed to above was actually not the one we were discussing 😬
Below are the 2 potential places where the calibration code (compute_Sv
) code can have bottleneck:
in get_vend_cal_params_power
where a broadcasting is probably used to get a data variable into the "right" shape based on indexing:
https://github.com/OSOceanAcoustics/echopype/blob/593a7e173dffd34858a7587cf2f31633a23a841c/echopype/calibrate/cal_params.py#L299-L304
in harmonize_env_param_time
where there is either a check for all the timestamps or an interpolation:
https://github.com/OSOceanAcoustics/echopype/blob/593a7e173dffd34858a7587cf2f31633a23a841c/echopype/calibrate/env_params.py#L53-L66
@anantmittal : Thanks for investigating the xr.where
in the masking code though -- since the change is to use dask.where
, the solution would probably be similarly useful for the get_vend_cal_params_power
case above.
I'll close this now, since my comment above is replicated in #1200.
This
xr.where
needs to be investigated to see if it can parallelize with the use of dask.https://github.com/OSOceanAcoustics/echopype/blob/3ca028f04a4fd3643bb0ad350dc09a699883ee1e/echopype/mask/api.py#L500