Closed iaraota closed 3 months ago
Attention: Patch coverage is 83.33333%
with 2 lines
in your changes missing coverage. Please review.
Project coverage is 49.62%. Comparing base (
720bd0c
) to head (922827c
). Report is 13 commits behind head on master.
Files | Patch % | Lines |
---|---|---|
gwsumm/data/coherence.py | 75.00% | 1 Missing :warning: |
gwsumm/data/spectral.py | 87.50% | 1 Missing :warning: |
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
@iaraota thanks! I was wondering: if I understand correctly, this check handles the possibility of new incoming spectrogram data to be added to the global memory variable will be excluded, but what about data already in the archive? Is the point that the corrupted data would never have made it into the archive file in the first place because of this check?
@iaraota thanks! I was wondering: if I understand correctly, this check handles the possibility of new incoming spectrogram data to be added to the global memory variable will be excluded, but what about data already in the archive? Is the point that the corrupted data would never have made it into the archive file in the first place because of this check?
This change prevents nan
values from entering the archive. Although we could add a check in the archive to delete nan
data, this additional step may be unnecessary since this change already addresses the issue. Instead, we could simply delete the archive of the broken pages and rerun. Do we want to implement this extra step?
Thanks for clarifying. I think this solution is sufficient. However I'm wondering about another impact from excluding data from the global memory variable: is it possible that some channel(s) may have some nan
value and thus the spectrogram shape is different than other spectrograms. Is it possible there could be some downstream consequence of that (like computing coherences or anything like that)? Are there other data elements that could be impacted by nan
values?
This function is not used in the coherence.py
, so it should not be a problem in the coherence computation.
This function is not used in the
coherence.py
, so it should not be a problem in the coherence computation.
Which function? I'm also wondering if other computations (like coherence) would be impacted by some channels having nan
but not others. Should coherence-grams also be protected in the same way as spectrograms?
This function is not used in the
coherence.py
, so it should not be a problem in the coherence computation.Which function? I'm also wondering if other computations (like coherence) would be impacted by some channels having
nan
but not others. Should coherence-grams also be protected in the same way as spectrograms?
Maybe I miss understood you. We can add this check after line 288.
@eagoetz updated the PR to also ignore nan
in coherence spectrograms.
@eagoetz updated the PR to also ignore
nan
in coherence spectrograms.
Awesome, thank you! One more minor suggestion: can a warning be printed so that it is more obvious when something like this happens so that it is not silently hidden from the user or log files? Thanks!
@eagoetz updated the PR with warnings.
If the data is corrupted, the computed spectrograms may contain
nan
values. While thesenan
values are ignored in spectrogram plots, but they can cause issues in derived products such as the power spectral density and ratio spectrogram, where the presence ofnan
values leads to all values becomingnan
, resulting in empty plots. This pull request addresses this issue by removing any potentialnan
values from the data used and saved inglobalv.SPECTROGRAMS
.