claritychallenge / clarity

Clarity Challenge toolkit - software for building Clarity Challenge systems
https://claritychallenge.github.io/clarity
MIT License
129 stars 54 forks source link

Bringing CAD1 recipe from v0.3.4 to main #329

Closed groadabike closed 11 months ago

groadabike commented 1 year ago

close #319

This PR implements all improvements made in CAD1 from version 0.3.3 to the main branch to avoid branches from diverging. It now includes changes from version 0.3.4

codecov[bot] commented 1 year ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (dd866c1) 92.86% compared to head (c6d6afc) 92.88%. Report is 6 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #329 +/- ## ========================================== + Coverage 92.86% 92.88% +0.02% ========================================== Files 43 43 Lines 3923 3921 -2 ========================================== - Hits 3643 3642 -1 + Misses 280 279 -1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

groadabike commented 1 year ago

@jonbarker68 This PR are the changes we made for CAD 1 to bring it to MAIN

groadabike commented 1 year ago

@jonbarker68 is ready for review. It is bringing version 0.3.4 to the main branch

Note that in ICASSP and here I am using a new class for the results clarity/utils/results_support.py I created this class to avoid creating the same class in all evaluate.py where the only difference is the header's column names. The header can be a param and the values can be sent in a dict

groadabike commented 1 year ago

It failed running Python 3.8 in test

=========================== short test summary info ============================
FAILED tests/regression/test_predictors.py::test_torch_msbg_stoi_xeon_e5_2673_cpu

I added to the merge queue again

groadabike commented 1 year ago

The second time, it was an urllib library error

=========================== short test summary info ============================
FAILED tests/recipes/cad1/task1/baseline/test_enhance_task1.py::test_decompose_signal[separation_model1-True] - urllib.error.HTTPError: HTTP Error 403: FORBIDDEN
===== 1 failed, 490 passed, 22 skipped, 4404 warnings in 152.30s (0:02:32) =====

The error happened when downloading the separation model from Pytorch hub in decompose signal

model = torch.hub.load("sigsep/open-unmix-pytorch", "umxhq").double()

resubmitting to the queue to see if the error happens again or was a one-time event

groadabike commented 1 year ago

@jonbarker68 The errors are happening when requesting the open-unmix model from torch.hub. It is a 403 error. The error is only in the workflow in GitHub, locally I can't see the issue. I am testing what happens if I apply this workaround https://github.com/pytorch/pytorch/issues/61755#issuecomment-885801511

groadabike commented 11 months ago

@jonbarker68 This PR was having a weird error when downloading a model from torch hub. I included a mark.xfail decorator. We will need to check in detail later why are we having this error with torch hub