claritychallenge / clarity

Clarity Challenge toolkit - software for building Clarity Challenge systems
https://claritychallenge.github.io/clarity
MIT License
121 stars 51 forks source link

Baseline for ICASSP 2024 Cadenza Challenge #340

Closed groadabike closed 1 year ago

groadabike commented 1 year ago

Closes #339

This PR should result in pyclarity v0.4.0

groadabike commented 1 year ago

@jonbarker68, @ns-rse The tests in this Draft PR are failing in 2 unrelated tests. Are we having the same issue as before with different architectures? :fearful:


    def test_smear3():
        """Test smear3"""
        np.random.seed(0)
        input_signal = np.random.rand(10000)
        smear_mat = make_smear_mat3(rl=R_LOWER, ru=R_UPPER, sample_rate=SAMPLE_RATE)
        output_signal = smear3(smear_mat, input_signal)
        assert output_signal.shape == (10240,)
>       assert np.sum(np.abs(output_signal)) == pytest.approx(
            5066.986397433977, rel=pytest.rel_tolerance, abs=pytest.abs_tolerance
        )
E       assert 5047.791163861037 == 5066.986397433977 ± 5.1e-04
E         comparison failed
E         Obtained: 5047.79[116](https://github.com/claritychallenge/clarity/actions/runs/6150288476/job/16687922487?pr=340#step:7:117)3861037
E         Expected: 5066.986397433977 ± 5.1e-04

tests/evaluator/msbg/test_smearing.py:41: AssertionError
____________________________ test_smear_valid_input ____________________________

    def test_smear_valid_input():
        """Test smear with valid input"""
        np.random.seed(0)
        input_signal = np.random.rand(10000)
        output_signal = Smearer(rl=R_LOWER, ru=R_UPPER, sample_rate=SAMPLE_RATE).smear(
            input_signal
        )
        assert output_signal.shape == (10240,)
>       assert np.sum(np.abs(output_signal)) == pytest.approx(
            5066.986397433977, rel=pytest.rel_tolerance, abs=pytest.abs_tolerance
        )
E       assert 5047.791163861037 == 5066.986397433977 ± 5.1e-04
E         comparison failed
E         Obtained: 5047.791163861037
E         Expected: 5066.986397433977 ± 5.1e-04

tests/evaluator/msbg/test_smearing.py:67: AssertionError
groadabike commented 1 year ago

@jonbarker68 Test files included

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 88.35% and project coverage change: -0.18% :warning:

Comparison is base (1663f32) 92.95% compared to head (7dc037e) 92.78%.

:exclamation: Current head 7dc037e differs from pull request most recent head 92b444f. Consider uploading reports for the commit 92b444f to get more accurate results

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #340 +/- ## ========================================== - Coverage 92.95% 92.78% -0.18% ========================================== Files 40 43 +3 Lines 3778 3924 +146 ========================================== + Hits 3512 3641 +129 - Misses 266 283 +17 ``` | [Files Changed](https://app.codecov.io/gh/claritychallenge/clarity/pull/340?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Δ | | |---|---|---| | [clarity/evaluator/haaqi/haaqi.py](https://app.codecov.io/gh/claritychallenge/clarity/pull/340?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-Y2xhcml0eS9ldmFsdWF0b3IvaGFhcWkvaGFhcWkucHk=) | `90.47% <ø> (ø)` | | | [clarity/utils/flac\_encoder.py](https://app.codecov.io/gh/claritychallenge/clarity/pull/340?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-Y2xhcml0eS91dGlscy9mbGFjX2VuY29kZXIucHk=) | `84.05% <84.05%> (ø)` | | | [clarity/utils/source\_separation\_support.py](https://app.codecov.io/gh/claritychallenge/clarity/pull/340?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-Y2xhcml0eS91dGlscy9zb3VyY2Vfc2VwYXJhdGlvbl9zdXBwb3J0LnB5) | `85.71% <85.71%> (ø)` | | | [clarity/utils/results\_support.py](https://app.codecov.io/gh/claritychallenge/clarity/pull/340?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-Y2xhcml0eS91dGlscy9yZXN1bHRzX3N1cHBvcnQucHk=) | `100.00% <100.00%> (ø)` | | | [clarity/utils/signal\_processing.py](https://app.codecov.io/gh/claritychallenge/clarity/pull/340?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-Y2xhcml0eS91dGlscy9zaWduYWxfcHJvY2Vzc2luZy5weQ==) | `97.43% <100.00%> (+0.88%)` | :arrow_up: |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.