gwpy / gwsumm

Gravitational-wave interferometer summary information system
GNU General Public License v3.0
12 stars 23 forks source link

Fix coherence spectrogram archive problem #406

Closed iaraota closed 5 months ago

iaraota commented 5 months ago

This PR addresses the issue encountered when computing coherence, which leads to a ValueError due to incompatible shapes, as illustrated in the following example:

ValueError: operands could not be broadcast together with shapes (734,8193) (9,8193) 

This problem often arises from utilizing the same channel across different coherence plots. It may occur when additional data is available during the computation of another plot, causing discrepancies in the segments of globalv.COHERENCE_COMPONENTS for that channel and the previously computed channel when saved to the HDF5 archive.

To tackle this issue, this PR ensures that all coherence components used have consistent segments within the globalv.COHERENCE_COMPONENTS variable.

Testing this PR

To test this PR copy the corrupted archive located in /home/iara.ota/public_html/summary/day/20240410/archive/L1-LSC_COHERENCE-1396742418-86400.h5_bkp to the local /home/${USER}/public_html/summary/day/20240410/archive folder and rename it to remove the _bkp suffix. Run the GWSumm process for the LSC Coherence tab with the detchar conda environment to make sure the archive is corrupted and then with this PR changes:

python -m gwsumm day --verbose --ifo L1 --on-segdb-error warn --on-datafind-error warn --output-dir /home/iara.ota/public_html/summary --config-file /home/detchar/public_html/summary/etc/defaults.ini,/home/detchar/public_html/summary/etc/global.ini,/home/iara.ota/src/ligo-summary-pages/configurations/common/lsc-coherence.ini --multi-process 32 --archive LSC_COHERENCE 20240410

Run example: https://ldas-jobs.ligo-la.caltech.edu/~iara.ota/summary/day/20240410/isc/coherence/

codecov[bot] commented 5 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 49.63%. Comparing base (720bd0c) to head (9efe8f9). Report is 4 commits behind head on master.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #406 +/- ## ========================================== - Coverage 49.64% 49.63% -0.02% ========================================== Files 60 60 Lines 8823 8840 +17 ========================================== + Hits 4380 4387 +7 - Misses 4443 4453 +10 ``` | [Flag](https://app.codecov.io/gh/gwpy/gwsumm/pull/406/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=gwpy) | Coverage Δ | | |---|---|---| | [Linux](https://app.codecov.io/gh/gwpy/gwsumm/pull/406/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=gwpy) | `49.63% <100.00%> (-0.02%)` | :arrow_down: | | [macOS](https://app.codecov.io/gh/gwpy/gwsumm/pull/406/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=gwpy) | `49.63% <100.00%> (?)` | | | [python3.10](https://app.codecov.io/gh/gwpy/gwsumm/pull/406/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=gwpy) | `49.63% <100.00%> (-0.02%)` | :arrow_down: | | [python3.11](https://app.codecov.io/gh/gwpy/gwsumm/pull/406/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=gwpy) | `49.63% <100.00%> (-0.02%)` | :arrow_down: | | [python3.9](https://app.codecov.io/gh/gwpy/gwsumm/pull/406/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=gwpy) | `49.63% <100.00%> (-0.02%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=gwpy#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

iaraota commented 5 months ago

@eagoetz thank you for the suggestions. The running example is the same link as before: https://ldas-jobs.ligo-la.caltech.edu/~iara.ota/summary/day/20240410/isc/coherence/