search

jongwook / onsets-and-frames

A Pytorch implementation of Onsets and Frames (Hawthorne 2018)
MIT License
208 stars 65 forks source link

Errors in mir_eval library #1

Closed yuanshenli closed 5 years ago

yuanshenli commented 5 years ago

I'm running with the following parameters in train.py

batch_size = 4
sequence_length = 327680
model_complexity = 16

And I got the following error:

0%|          | 499/500000 [01:32<25:51:47,  5.36it/s]/home/lab/.linuxbrew/Cellar/python/3.7.2_2/lib/python3.7/site-packages/mir_eval/transcription.py:167: UserWarning: Estimated notes are empty.
  warnings.warn("Estimated notes are empty.")
/home/lab/.linuxbrew/Cellar/python/3.7.2_2/lib/python3.7/site-packages/mir_eval/multipitch.py:275: UserWarning: Estimate frequencies are all empty.
  warnings.warn("Estimate frequencies are all empty.")
  1%|          | 2999/500000 [10:07<25:37:36,  5.39it/s]/home/lab/.linuxbrew/Cellar/python/3.7.2_2/lib/python3.7/site-packages/mir_eval/transcription_velocity.py:185: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  ref_matched_velocities)[0]
ERROR - train_transcriber - Failed after 0:10:13!
Traceback (most recent calls WITHOUT Sacred internals):
  File "train.py", line 117, in train
    for key, value in evaluate(validation_dataset, model).items():
  File "/home/lab/Documents/CS224NProject/Sample/onsets-and-frames/evaluate.py", line 70, in evaluate
    p, r, f, o = evaluate_notes_with_velocity(i_ref, p_ref, v_ref, i_est, p_est, v_est, velocity_tolerance=0.1)
  File "/home/lab/.linuxbrew/Cellar/python/3.7.2_2/lib/python3.7/site-packages/mir_eval/transcription_velocity.py", line 291, in precision_recall_f1_overlap
    offset_min_tolerance, strict, velocity_tolerance)
  File "/home/lab/.linuxbrew/Cellar/python/3.7.2_2/lib/python3.7/site-packages/mir_eval/transcription_velocity.py", line 178, in match_notes
    ref_matched_velocities = ref_velocities[matching[:, 0]]
IndexError: too many indices for array

BTW, another issue, I first tried all the default parameters, and I kept getting GPU out of memory message. My system should meet the minimum system requirements.

Does anyone have insights into these two issues?

jongwook commented 5 years ago

Hi @yuanshenli , This is because of an mir_eval issue where it fails when there is no matching which was only recently fixed. This no-matching situation inevitably happens in early iterations. There is a recent patch merged to mir_eval to fix this, but this hasn't made it through a new release at the moment.

So for now, I'd recommend installing the latest commit of mir_eval by:

pip uninstall mir_eval
pip install git+https://github.com/craffel/mir_eval.git

in order to install the most recent version of the mir_eval package out of the master branch.

After doing this, the training should run just fine; let me know if you encounter any issues.

yuanshenli commented 5 years ago

Hi @yuanshenli , This is because of an mir_eval issue where it fails when there is no matching which was only recently fixed. This no-matching situation inevitably happens in early iterations. There is a recent path merged to mir_eval to fix this, but this hasn't made it through a new release at the moment.

So for now, I'd recommend installing the latest commit of mir_eval by:

pip uninstall mir_eval
pip install git+https://github.com/craffel/mir_eval.git

in order to install the most recent version of the mir_eval package out of the master branch.

After doing this, the training should run just fine; let me know if you encounter any issues.

Thank you so much for the timely reply! so far it made through the first 2000 iterations, it did get the warning warnings.warn("Estimated notes are empty.") at first 500 iterations like before, but as you suggested it just happed in early iterations.

May I ask what values you used for batch_size and model_complexity? I kept getting cuda out_of_memory error. below is my gpu configuration:

Mon Mar  4 18:32:31 2019       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 390.77                 Driver Version: 390.77                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX TIT...  Off  | 00000000:02:00.0 Off |                  N/A |
| 57%   83C    P2    89W / 250W |    525MiB / 12212MiB |      4%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 108...  Off  | 00000000:03:00.0 Off |                  N/A |
| 29%   60C    P2   133W / 250W |   2951MiB / 11178MiB |     84%      Default |
+-------------------------------+----------------------+----------------------+
|   2  GeForce GTX 108...  Off  | 00000000:04:00.0  On |                  N/A |
|  0%   48C    P8    14W / 280W |    657MiB / 11175MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
yuanshenli commented 5 years ago

Hi @yuanshenli , This is because of an mir_eval issue where it fails when there is no matching which was only recently fixed. This no-matching situation inevitably happens in early iterations. There is a recent path merged to mir_eval to fix this, but this hasn't made it through a new release at the moment.

So for now, I'd recommend installing the latest commit of mir_eval by:

pip uninstall mir_eval
pip install git+https://github.com/craffel/mir_eval.git

in order to install the most recent version of the mir_eval package out of the master branch.

After doing this, the training should run just fine; let me know if you encounter any issues.

Just as I thought it was working, same error occurred:

  0%|          | 499/500000 [01:36<26:19:05,  5.27it/s]/home/lab/.linuxbrew/Cellar/python/3.7.2_2/lib/python3.7/site-packages/mir_eval/transcription.py:167: UserWarning: Estimated notes are empty.
  warnings.warn("Estimated notes are empty.")
/home/lab/.linuxbrew/Cellar/python/3.7.2_2/lib/python3.7/site-packages/mir_eval/multipitch.py:275: UserWarning: Estimate frequencies are all empty.
  warnings.warn("Estimate frequencies are all empty.")
  0%|          | 2499/500000 [09:13<26:18:47,  5.25it/s]/home/lab/.linuxbrew/Cellar/python/3.7.2_2/lib/python3.7/site-packages/mir_eval/transcription_velocity.py:185: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  ref_matched_velocities)[0]
ERROR - train_transcriber - Failed after 0:09:22!
Traceback (most recent calls WITHOUT Sacred internals):
  File "train.py", line 117, in train
    for key, value in evaluate(validation_dataset, model).items():
  File "/home/lab/Documents/CS224NProject/Sample/onsets-and-frames/evaluate.py", line 70, in evaluate
    p, r, f, o = evaluate_notes_with_velocity(i_ref, p_ref, v_ref, i_est, p_est, v_est, velocity_tolerance=0.1)
  File "/home/lab/.linuxbrew/Cellar/python/3.7.2_2/lib/python3.7/site-packages/mir_eval/transcription_velocity.py", line 291, in precision_recall_f1_overlap
    offset_min_tolerance, strict, velocity_tolerance)
  File "/home/lab/.linuxbrew/Cellar/python/3.7.2_2/lib/python3.7/site-packages/mir_eval/transcription_velocity.py", line 178, in match_notes
    ref_matched_velocities = ref_velocities[matching[:, 0]]
IndexError: too many indices for array
yuanshenli commented 5 years ago

Hi @yuanshenli , This is because of an mir_eval issue where it fails when there is no matching which was only recently fixed. This no-matching situation inevitably happens in early iterations. There is a recent path merged to mir_eval to fix this, but this hasn't made it through a new release at the moment. So for now, I'd recommend installing the latest commit of mir_eval by:

pip uninstall mir_eval
pip install git+https://github.com/craffel/mir_eval.git

in order to install the most recent version of the mir_eval package out of the master branch. After doing this, the training should run just fine; let me know if you encounter any issues.

Just as I thought it was working, same error occurred:

  0%|          | 499/500000 [01:36<26:19:05,  5.27it/s]/home/lab/.linuxbrew/Cellar/python/3.7.2_2/lib/python3.7/site-packages/mir_eval/transcription.py:167: UserWarning: Estimated notes are empty.
  warnings.warn("Estimated notes are empty.")
/home/lab/.linuxbrew/Cellar/python/3.7.2_2/lib/python3.7/site-packages/mir_eval/multipitch.py:275: UserWarning: Estimate frequencies are all empty.
  warnings.warn("Estimate frequencies are all empty.")
  0%|          | 2499/500000 [09:13<26:18:47,  5.25it/s]/home/lab/.linuxbrew/Cellar/python/3.7.2_2/lib/python3.7/site-packages/mir_eval/transcription_velocity.py:185: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  ref_matched_velocities)[0]
ERROR - train_transcriber - Failed after 0:09:22!
Traceback (most recent calls WITHOUT Sacred internals):
  File "train.py", line 117, in train
    for key, value in evaluate(validation_dataset, model).items():
  File "/home/lab/Documents/CS224NProject/Sample/onsets-and-frames/evaluate.py", line 70, in evaluate
    p, r, f, o = evaluate_notes_with_velocity(i_ref, p_ref, v_ref, i_est, p_est, v_est, velocity_tolerance=0.1)
  File "/home/lab/.linuxbrew/Cellar/python/3.7.2_2/lib/python3.7/site-packages/mir_eval/transcription_velocity.py", line 291, in precision_recall_f1_overlap
    offset_min_tolerance, strict, velocity_tolerance)
  File "/home/lab/.linuxbrew/Cellar/python/3.7.2_2/lib/python3.7/site-packages/mir_eval/transcription_velocity.py", line 178, in match_notes
    ref_matched_velocities = ref_velocities[matching[:, 0]]
IndexError: too many indices for array

Hmm just realized that my mir_eval wasn't updated properly, let me try again.