search

bytedance / piano_transcription

1.67k stars 197 forks source link

Python is not installed as a framework? what does this mean? #1

Open achillis2 opened 4 years ago

achillis2 commented 4 years ago

I got the following error when I tried to use the package. Can you help resolve it? Thanks!

Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 05:52:31) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information.

from piano_transcription_inference import PianoTranscription, sample_rate, load_audio Traceback (most recent call last): File "", line 1, in File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/piano_transcription_inference/init.py", line 1, in from .inference import PianoTranscription File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/piano_transcription_inference/inference.py", line 11, in from .models import Regress_onset_offset_frame_velocity_CRNN, Note_pedal File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/piano_transcription_inference/models.py", line 6, in import matplotlib.pyplot as plt File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/matplotlib/pyplot.py", line 2372, in switch_backend(rcParams["backend"]) File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/matplotlib/pyplot.py", line 207, in switch_backend backend_mod = importlib.import_module(backend_name) File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/matplotlib/backends/backend_macosx.py", line 14, in from matplotlib.backends import _macosx ImportError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are using (Ana)Conda please install python.app and replace the use of 'python' with 'pythonw'. See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more information.

achillis2 commented 4 years ago

Never mind. This link below solves the issue.

https://stackoverflow.com/questions/21784641/installation-issue-with-matplotlib-python

qiuqiangkong commented 4 years ago

Hi, do you have Python3 installed? Try $ python3 in your terminal and see what you get?

On Tue, 1 Dec 2020 at 02:37, achillis2 notifications@github.com wrote:

I got the following error when I tried to use the package. Can you help resolve it? Thanks!

Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 05:52:31) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information.

from piano_transcription_inference import PianoTranscription, sample_rate, load_audio Traceback (most recent call last): File "", line 1, in File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/piano_transcription_inference/ init.py", line 1, in from .inference import PianoTranscription File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/piano_transcription_inference/inference.py", line 11, in from .models import Regress_onset_offset_frame_velocity_CRNN, Note_pedal File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/piano_transcription_inference/models.py", line 6, in import matplotlib.pyplot as plt File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/matplotlib/pyplot.py", line 2372, in switch_backend(rcParams["backend"]) File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/matplotlib/pyplot.py", line 207, in switch_backend backend_mod = importlib.import_module(backend_name) File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/importlib/ init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/matplotlib/backends/backend_macosx.py", line 14, in from matplotlib.backends import _macosx ImportError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are using (Ana)Conda please install python.app and replace the use of 'python' with 'pythonw'. See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more information.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSK74COSOS3JR6B7OR3SSPRAJANCNFSM4UH6LPDA .

GUOhm230 commented 4 years ago

只能精确到5秒嘛？如果做音频的实时转录，5秒时间有点太长了，以2秒为切割效果不好，请问需要改哪些地方呢

qiuqiangkong commented 4 years ago

这个repo是离线系统，如果实时的话把RNN的bidirectional去掉就可以实时了

GUOhm230 commented 3 years ago

这个repo是离线系统，如果实时的话把RNN的bidirectional去掉就可以实时了

作者大佬您好，再打扰一下，目前在读研一。看到您的论文，又有代码十分的友好，因此想学习一下。冒昧打扰，希望您可以为我解答，十分感谢; 1)我所问的实时是指一段音频作为输入，刚演奏出一个音符，能立即识别出该音符是什么音高，请问这个可以实现吗？ 2)去掉RNN的双向部分可以实现实时是什么原理，我不太明白，希望可以得到您简短的解答。

qiuqiangkong commented 3 years ago

您好！

1) 目前的代码为了达到最好的效果，所以不是实时的模型，如果需要搭建实时模型，需要改动代码重新训练。

2) 双向指的是用历史和未来信息，要实时的话，不能使用未来信息，所以改成只用历史信息的单向RNN。

Best wishes， Qiuqiang

GUOhm230 notifications@github.com于2020年12月8日 周二11:51写道：

这个repo是离线系统，如果实时的话把RNN的bidirectional去掉就可以实时了

作者大佬您好，再打扰一下，目前在读研一。看到您的论文，又有代码十分的友好，因此想学习一下。冒昧打扰，希望您可以为我解答，十分感谢; 1)我所问的实时是指一段音频作为输入，刚演奏出一个音符，能立即识别出该音符是什么音高，请问这个可以实现吗？ 2)去掉RNN的双向部分可以实现实时是什么原理，我不太明白，希望可以得到您简短的解答。

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-740357155, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSOQKWPXCHXLH4ETODLSTWPDTANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好，再次打扰，十分抱歉： 我按照您的文档，目前正在训练你的模型，由于迟迟没有进入迭代，因此有些疑惑； 1.请问一开始耗费时间过长都是在处理数据是嘛？时间长达10小时。终端显示部分内容如下：

>  _root        : INFO     Namespace(augmentation='none', batch_size=12, cuda=True, early_stop=300000, filename='main', learning_rate=0.0005, loss_type='regress_pedal_bce', max_note_shift=0, mini_data=False, mode='train', model_type='Regress_pedal_CRNN', reduce_iteration=10000, resume_iteration=0, workspace='./workspaces/piano_transcription')
> root        : INFO     Using GPU.
> root        : INFO     train segments: 571589
> root        : INFO     Evaluate train segments: 571589
> root        : INFO     Evaluate validation segments: 68646
> root        : INFO     Evaluate test segments: 71959_
      2.在数据处理的过程中报错如下，我用的仍然是您的数据。如果有空的话，希望作者可以花个两分钟帮我看一下，十分感谢：

>   _```
> File "/home/mnc03/anaconda3/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 202, in forward
>     self.padding, self.dilation, self.groups)
> RuntimeError: cuDNN error: CUDNN_STATUS_MAPPING_ERROR
> 
> /home/mnc03/anaconda3/lib/python3.7/site-packages/sklearn/feature_extraction/text.py:17: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
>   from collections import Mapping, defaultdict
> concurrent.futures.process._RemoteTraceback:
> """
> Traceback (most recent call last):
>   File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/process.py", line 239, in _process_worker
>     r = call_item.fn(*call_item.args, **call_item.kwargs)
>   File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/process.py", line 198, in _process_chunk
>     return [fn(*args) for args in chunk]
>   File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/process.py", line 198, in <listcomp>
>     return [fn(*args) for args in chunk]
>   File "pytorch/calculate_score_for_paper.py", line 203, in calculate_score_per_song
>     total_dict = pickle.load(open(prob_path, 'rb'))
> FileNotFoundError: [Errno 2] No such file or directory: './workspaces/piano_transcription/probs/model_type=Note_pedal/augmentation=aug/dataset=maestro/split=test/MIDI-Unprocessed_11_R3_2008_01-04_ORIG_MID--AUDIO_11_R3_2008_wav--1.pkl'
> """
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "pytorch/calculate_score_for_paper.py", line 385, in <module>
>     calculate_metrics(args)
>   File "pytorch/calculate_score_for_paper.py", line 350, in calculate_metrics
>     stats_dict = score_calculator.metrics(thresholds)
>   File "pytorch/calculate_score_for_paper.py", line 182, in metrics
>     stats_list = list(results)
>   File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/process.py", line 483, in _chain_from_iterable_of_lists
>     for element in iterable:
>   File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 598, in result_iterator
>     yield fs.pop().result()
>   File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 428, in result
>     return self.__get_result()
>   File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
>     raise self._exception
> FileNotFoundError: [Errno 2] No such file or directory: './workspaces/piano_transcription/probs/model_type=Note_pedal/augmentation=aug/dataset=maestro/split=test/MIDI-Unprocessed_11_R3_2008_01-04_ORIG_MID--AUDIO_11_R3_2008_wav--1.pkl'
> /home/mnc03/anaconda3/lib/python3.7/site-packages/sklearn/feature_extraction/text.py:17: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
>   from collections import Mapping, defaultdict
> Traceback (most recent call last):
>   File "pytorch/calculate_score_for_paper.py", line 385, in <module>
>     calculate_metrics(args)
>   File "pytorch/calculate_score_for_paper.py", line 350, in calculate_metrics
>     stats_dict = score_calculator.metrics(thresholds)
>   File "pytorch/calculate_score_for_paper.py", line 184, in metrics
>     for key in stats_list[0].keys():
> IndexError: list index out of range
> ```_
qiuqiangkong commented 3 years ago

您好！

 先把

https://github.com/bytedance/piano_transcription/blob/master/pytorch/main.py#L199 到 https://github.com/bytedance/piano_transcription/blob/master/pytorch/main.py#L241 注释掉试试看能不能跑？

On Thu, 17 Dec 2020 at 10:45, GUOhm230 notifications@github.com wrote:

作者您好，再次打扰，十分抱歉： 我按照您的文档，目前正在训练你的模型，由于迟迟没有进入迭代，因此有些疑惑； 1.请问一开始耗费时间过长都是在处理数据是嘛？时间长达10小时。终端显示部分内容如下：

_root : INFO Namespace(augmentation='none', batch_size=12, cuda=True, early_stop=300000, filename='main', learning_rate=0.0005, loss_type='regress_pedal_bce', max_note_shift=0, mini_data=False, mode='train', model_type='Regress_pedal_CRNN', reduce_iteration=10000, resume_iteration=0, workspace='./workspaces/piano_transcription')

root : INFO Using GPU.

root : INFO train segments: 571589

root : INFO Evaluate train segments: 571589

root : INFO Evaluate validation segments: 68646

root : INFO Evaluate test segments: 71959_

  2.在数据处理的过程中报错如下，我用的仍然是您的数据。如果有空的话，希望作者可以花个两分钟帮我看一下，十分感谢：

_```

File "/home/mnc03/anaconda3/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 202, in forward

self.padding, self.dilation, self.groups)

RuntimeError: cuDNN error: CUDNN_STATUS_MAPPING_ERROR

/home/mnc03/anaconda3/lib/python3.7/site-packages/sklearn/feature_extraction/text.py:17: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working

from collections import Mapping, defaultdict

concurrent.futures.process._RemoteTraceback:

"""

Traceback (most recent call last):

File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/process.py", line 239, in _process_worker

r = call_item.fn(*call_item.args, **call_item.kwargs)

File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/process.py", line 198, in _process_chunk

return [fn(*args) for args in chunk]

File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/process.py", line 198, in 

return [fn(*args) for args in chunk]

File "pytorch/calculate_score_for_paper.py", line 203, in calculate_score_per_song

total_dict = pickle.load(open(prob_path, 'rb'))

FileNotFoundError: [Errno 2] No such file or directory: './workspaces/piano_transcription/probs/model_type=Note_pedal/augmentation=aug/dataset=maestro/split=test/MIDI-Unprocessed_11_R3_2008_01-04_ORIG_MID--AUDIO_11_R3_2008_wav--1.pkl'

"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "pytorch/calculate_score_for_paper.py", line 385, in 

calculate_metrics(args)

File "pytorch/calculate_score_for_paper.py", line 350, in calculate_metrics

stats_dict = score_calculator.metrics(thresholds)

File "pytorch/calculate_score_for_paper.py", line 182, in metrics

stats_list = list(results)

File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/process.py", line 483, in _chain_from_iterable_of_lists

for element in iterable:

File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 598, in result_iterator

yield fs.pop().result()

File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 428, in result

return self.__get_result()

File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result

raise self._exception

FileNotFoundError: [Errno 2] No such file or directory: './workspaces/piano_transcription/probs/model_type=Note_pedal/augmentation=aug/dataset=maestro/split=test/MIDI-Unprocessed_11_R3_2008_01-04_ORIG_MID--AUDIO_11_R3_2008_wav--1.pkl'

/home/mnc03/anaconda3/lib/python3.7/site-packages/sklearn/feature_extraction/text.py:17: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working

from collections import Mapping, defaultdict

Traceback (most recent call last):

File "pytorch/calculate_score_for_paper.py", line 385, in 

calculate_metrics(args)

File "pytorch/calculate_score_for_paper.py", line 350, in calculate_metrics

stats_dict = score_calculator.metrics(thresholds)

File "pytorch/calculate_score_for_paper.py", line 184, in metrics

for key in stats_list[0].keys():

IndexError: list index out of range

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-747168280, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSI4XXTFU7VJEFHGWNLSVFWEPANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

十分感谢您的建议： 按照您的方法注释，仍然是同样的问题。我怕数据被我弄坏了。又把数据重新下载了一遍。现仍然有如下报错： 1：File "pytorch/main.py", line 297, in train(args) File "pytorch/main.py", line 253, in train batch_output_dict = model(batch_data_dict['waveform']) File "/home/mnc03/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in call ..... RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED 这个问题官方说是，没有在模型中添加GPU，于是我在main中加入model = model(gpu).问题依旧 2**RuntimeError: Caught RuntimeError in replica 0 on device 0.** 百度后添加 import os os.environ["CUDA_VISIBLE_DEVICES"] = "0,2,3"。仍然出错。 3.（同上一个问题）The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "pytorch/calculate_score_for_paper.py", line 385, in calculate_metrics(args) File "pytorch/calculate_score_for_paper.py", line 350, in calculate_metrics stats_dict = score_calculator.metrics(thresholds) File "pytorch/calculate_score_for_paper.py", line 182, in metrics stats_list = list(results) File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/process.py", line 483, in _chain_from_iterable_of_lists for element in iterable: File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 598, in result_iterator yield fs.pop().result() File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 428, in result return self.get_result() File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 384, in get_result raise self._exception FileNotFoundError: [Errno 2] No such file or directory: './workspaces/piano_transcription/probs/model_type=Note_pedal/augmentation=aug/dataset=maestro/split=test/MIDI-Unprocessed_11_R3_2008_01-04_ORIG_MID--AUDIO_11_R3_2008_wav--1.pkl' /home/mnc03/anaconda3/lib/python3.7/site-packages/sklearn/feature_extraction/text.py:17: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working from collections import Mapping, defaultdict Traceback (most recent call last): File "pytorch/calculate_score_for_paper.py", line 385, in calculate_metrics(args) File "pytorch/calculate_score_for_paper.py", line 350, in calculate_metrics stats_dict = score_calculator.metrics(thresholds) File "pytorch/calculate_score_for_paper.py", line 184, in metrics for key in stats_list[0].keys(): IndexError: list index out of range 解决无果。一直没办法自己训练这个模型。请作者花几分钟指教。十分感谢

qiuqiangkong commented 3 years ago

Hi, did you do pip install piano_transcription_inference?

On Tue, 1 Dec 2020 at 02:37, achillis2 notifications@github.com wrote:

I got the following error when I tried to use the package. Can you help resolve it? Thanks!

Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 05:52:31) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information.

from piano_transcription_inference import PianoTranscription, sample_rate, load_audio Traceback (most recent call last): File "", line 1, in File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/piano_transcription_inference/ init.py", line 1, in from .inference import PianoTranscription File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/piano_transcription_inference/inference.py", line 11, in from .models import Regress_onset_offset_frame_velocity_CRNN, Note_pedal File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/piano_transcription_inference/models.py", line 6, in import matplotlib.pyplot as plt File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/matplotlib/pyplot.py", line 2372, in switch_backend(rcParams["backend"]) File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/matplotlib/pyplot.py", line 207, in switch_backend backend_mod = importlib.import_module(backend_name) File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/importlib/ init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/matplotlib/backends/backend_macosx.py", line 14, in from matplotlib.backends import _macosx ImportError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are using (Ana)Conda please install python.app and replace the use of 'python' with 'pythonw'. See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more information.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSK74COSOS3JR6B7OR3SSPRAJANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

Hi, did you do pip install piano_transcription_inference? … On Tue, 1 Dec 2020 at 02:37, achillis2 @.**> wrote: I got the following error when I tried to use the package. Can you help resolve it? Thanks! Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 05:52:31) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. from piano_transcription_inference import PianoTranscription, sample_rate, load_audio Traceback (most recent call last): File "", line 1, in File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/piano_transcription_inference/ init.py", line 1, in from .inference import PianoTranscription File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/piano_transcription_inference/inference.py", line 11, in from .models import Regress_onset_offset_frame_velocity_CRNN, Note_pedal File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/piano_transcription_inference/models.py", line 6, in import matplotlib.pyplot as plt File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/matplotlib/pyplot.py", line 2372, in switch_backend(rcParams["backend"]) File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/matplotlib/pyplot.py", line 207, in switch_backend backend_mod = importlib.import_module(backend_name) File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/importlib/ init*.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/matplotlib/backends/backend_macosx.py", line 14, in from matplotlib.backends import _macosx ImportError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are using (Ana)Conda please install python.app and replace the use of 'python' with 'pythonw'. See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more information. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSK74COSOS3JR6B7OR3SSPRAJANCNFSM4UH6LPDA .

是按照你这上面的解释做的

qiuqiangkong commented 3 years ago

看起来好像是matplotlib之类的包安装错误，试试使用了类似conda的环境来管理包？

On Mon, 21 Dec 2020 at 09:10, GUOhm230 notifications@github.com wrote:

Hi, did you do pip install piano_transcription_inference? … <#m-2403082816755397268> On Tue, 1 Dec 2020 at 02:37, achillis2 @.**> wrote: I got the following error when I tried to use the package. Can you help resolve it? Thanks! Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 05:52:31) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. from piano_transcription_inference import PianoTranscription, sample_rate, load_audio Traceback (most recent call last): File "", line 1, in File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/piano_transcription_inference/ init.py", line 1, in from .inference import PianoTranscription File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/piano_transcription_inference/inference.py", line 11, in from .models import Regress_onset_offset_frame_velocity_CRNN, Note_pedal File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/piano_transcription_inference/models.py", line 6, in import matplotlib.pyplot as plt File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/matplotlib/pyplot.py", line 2372, in switch_backend(rcParams["backend"]) File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/matplotlib/pyplot.py", line 207, in switch_backend backend_mod = importlib.import_module(backend_name) File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/importlib/ init*.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/Users/dli/projects/personal/piano_transcription/lib/python3.6/site-packages/matplotlib/backends/backend_macosx.py", line 14, in from matplotlib.backends import _macosx ImportError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are using (Ana)Conda please install python.app and replace the use of 'python' with 'pythonw'. See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more information. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1 https://github.com/bytedance/piano_transcription/issues/1>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSK74COSOS3JR6B7OR3SSPRAJANCNFSM4UH6LPDA .

是按照你这上面的解释做的

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-748705750, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSNCBEEACLEUTN3Q4R3SV2OAJANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 感谢您的回复。您可能回复了上面这位同学的提问。我的问题如下（2个cudnn和GPU报错，一个数据越线错误。提示前面的两个问题是因为最后一个问题导致。尝试解决无果才再次打扰您，不胜感激）： 1：File "pytorch/main.py", line 297, in train(args) File "pytorch/main.py", line 253, in train batch_output_dict = model(batch_data_dict['waveform']) File "/home/mnc03/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in call ..... RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED 这个问题官方说是，没有在模型中添加GPU，于是我在main中加入model = model(gpu).问题依旧 2RuntimeError: Caught RuntimeError in replica 0 on device 0. 百度后添加 import os os.environ["CUDA_VISIBLE_DEVICES"] = "0,2,3"。仍然出错。 3.（同上一个问题）The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "pytorch/calculate_score_for_paper.py", line 385, in calculate_metrics(args) File "pytorch/calculate_score_for_paper.py", line 350, in calculate_metrics stats_dict = score_calculator.metrics(thresholds) File "pytorch/calculate_score_for_paper.py", line 182, in metrics stats_list = list(results) File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/process.py", line 483, in _chain_from_iterable_of_lists for element in iterable: File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 598, in result_iterator yield fs.pop().result() File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 428, in result return self.get_result() File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 384, in get_result raise self._exception FileNotFoundError: [Errno 2] No such file or directory: './workspaces/piano_transcription/probs/model_type=Note_pedal/augmentation=aug/dataset=maestro/split=test/MIDI-Unprocessed_11_R3_2008_01-04_ORIG_MID--AUDIO_11_R3_2008_wav--1.pkl' /home/mnc03/anaconda3/lib/python3.7/site-packages/sklearn/feature_extraction/text.py:17: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working from collections import Mapping, defaultdict Traceback (most recent call last): File "pytorch/calculate_score_for_paper.py", line 385, in calculate_metrics(args) File "pytorch/calculate_score_for_paper.py", line 350, in calculate_metrics stats_dict = score_calculator.metrics(thresholds) File "pytorch/calculate_score_for_paper.py", line 184, in metrics for key in stats_list[0].keys(): IndexError: list index out of range 解决无果。一直没办法自己训练这个模型。请作者花几分钟指教。十分感谢

qiuqiangkong commented 3 years ago

GUOhm230您好！

在测试阶段可以试试1. 能否用CPU inference。2. 能否用GPU inference。在你的实验环境是否安装了CUDNN？

在训练阶段，试试：1. 把evaluate相关代码注释掉看能否运行？

On Mon, 21 Dec 2020 at 10:13, GUOhm230 notifications@github.com wrote:

作者您好：

感谢您的回复。您可能回复了上面这位同学的提问。我的问题如下（2个cudnn和GPU报错，一个数据越线错误。提示前面的两个问题是因为最后一个问题导致。尝试解决无果才再次打扰您，不胜感激）： 1：File "pytorch/main.py", line 297, in train(args) File "pytorch/main.py", line 253, in train batch_output_dict = model(batch_data_dict['waveform']) File "/home/mnc03/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in call ..... RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED 这个问题官方说是，没有在模型中添加GPU，于是我在main中加入model = model(gpu).问题依旧 2RuntimeError: Caught RuntimeError in replica 0 on device 0. 百度后添加 import os os.environ["CUDA_VISIBLE_DEVICES"] = "0,2,3"。仍然出错。 3.（同上一个问题）The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "pytorch/calculate_score_for_paper.py", line 385, in calculate_metrics(args) File "pytorch/calculate_score_for_paper.py", line 350, in calculate_metrics stats_dict = score_calculator.metrics(thresholds) File "pytorch/calculate_score_for_paper.py", line 182, in metrics stats_list = list(results) File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/process.py", line 483, in _chain_from_iterable_of_lists for element in iterable: File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 598, in result_iterator yield fs.pop().result() File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 428, in result return self.get_result() File "/home/mnc03/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 384, in get_result raise self._exception FileNotFoundError: [Errno 2] No such file or directory: './workspaces/piano_transcription/probs/model_type=Note_pedal/augmentation=aug/dataset=maestro/split=test/MIDI-Unprocessed_11_R3_2008_01-04_ORIG_MID--AUDIO_11_R3_2008_wav--1.pkl' /home/mnc03/anaconda3/lib/python3.7/site-packages/sklearn/feature_extraction/text.py:17: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working from collections import Mapping, defaultdict Traceback (most recent call last): File "pytorch/calculate_score_for_paper.py", line 385, in calculate_metrics(args) File "pytorch/calculate_score_for_paper.py", line 350, in calculate_metrics stats_dict = score_calculator.metrics(thresholds) File "pytorch/calculate_score_for_paper.py", line 184, in metrics for key in stats_list[0].keys(): IndexError: list index out of range 解决无果。一直没办法自己训练这个模型。请作者花几分钟指教。十分感谢

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-748718357, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSP5K2NSGTELXX3TCZ3SV2VMZANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 十分感谢您的悉心指导。通过重装实验室服务器的cuda和pytorch，已经把训练问题解决了，目前应该是在训练（占用了GPU,ubuntu终端输出：48075 tensor(0.21980,....)。 目前遇到的问题是训练模型pth比较大，而且不能做到实时识别单个音符后立即输出该音符。所以我有以下想法： 1.能否将其中一部分网络结构删除。比如双向的GRU都改成单向,或者减少隐层数等减少模型参数的方法。 2.您尝试过一些模型裁剪的方法以减少模型参数嘛？预测准确率可以达到90%以上吗？ 3.能否实现逐个音符的跟踪，识别一个音符后，当即输出该音符。 祝您事业顺利，身体健康！ GuoHM230

qiuqiangkong commented 3 years ago

你好！

  1. 可以的，需要重新训练模型。
  2. 可以把所有参数数目减少，比如feature maps数目减少。准确率可以达到90%以上。
  3. 可以，把GRU改成单向重新训练即可。

祝好 孔秋强

On Tue, 22 Dec 2020 at 11:03, GUOhm230 notifications@github.com wrote:

作者您好： 十分感谢您的悉心指导。通过重装实验室服务器的cuda和pytorch，已经把训练问题解决了，目前应该是在训练（占用了GPU,ubuntu终端输出：48075 tensor(0.21980,....)。 目前遇到的问题是训练模型pth比较大，而且不能做到实时识别单个音符后立即输出该音符。所以我有以下想法： 1.能否将其中一部分网络结构删除。比如双向的GRU都改成单向,或者减少隐层数等减少模型参数的方法。 2.您尝试过一些模型裁剪的方法以减少模型参数嘛？预测准确率可以达到90%以上吗？ 3.能否实现逐个音符的跟踪，识别一个音符后，当即输出该音符。 祝您事业顺利，身体健康！ GuoHM230

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-749311013, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSKILQTDAROUSZMXBZTSWAEAVANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 非常感谢您再次百忙之中抽出时间帮我答疑解惑，深感惶恐和荣幸。根据最新进度，仍然有些问题想打扰您宝贵的时间帮我答疑解惑。 1.昨天正在尝试做模型训练，由于输出形式和您给出的样板不一样，而且checkpoint文件夹中没有保存每次iteration的pth，不知道我的训练是否是对的，而且我没办法知道到底进行到哪一次了。输出结果如：48075 tensor(0.21980,....) 2.请问作者在代码中（pytorch\main.py）中是否没有加入您已经训练好得模型参数？（CRNN_note_F1=0.9677_pedal_F1=0.9186.pth）我在代码中没有看到。我如果更改了模型结构（比如双向改单向，减少隐层数）还能加载您的模型进行finetune嘛？如果加入的话，是否代码写法如下： Model = eval(model_type) model = Model(frames_per_second=frames_per_second, classes_num=classes_num) load_model = torch.load('./CRNN_note_F1=0.9677_pedal_F1=0.9186.pth') model.load_state_dict(load_model) 即可 3.您代码中的建立模型部分是我不太理解的： Model = eval(model_type) model = Model(frames_per_second=frames_per_second, classes_num=classes_num) 该语句好像并没有调用，pytorch中的models.py中的内容，那是怎么做到的呢？我百度无果，所以请教。 4.您昨天所说的，把所有参数数目减少，比如feature maps,准确率达到90%以上，您是否有尝试.我对于参数数目减少的具体操作不太明白，怎么减少？减少多少？依据是什么呢？ 5.之前我在训练时，将MP3文件处理成输入的hdf5文件时是24.9G，也包含了所有数据的MP3文件。而昨天多次训练发现处理后的数据变成了4.1G，比如2004原本有MP3文件132项，之前是132项全部转化成hdf5，而昨天之后只有其中的11项转成了。是否是在训练过程中会把数据删除还是合并？或者只保存训练数据，其中测试数据和评价数据不再保存？ 连续叨扰，而且问题繁琐给作者造成诸多不便，希望您可以理解。您的项目实在很有价值，我作为初学者，想把整个项目完整的彻底的捋一遍，每个问题都是自己思考后，百度后方敢请教于您。十分期待您的回复。谢谢 祝事业顺利 GUOhm230

qiuqiangkong commented 3 years ago

您好！

1. 看来是进行到了48075个iteration。repo中每20000次保存一次模型：

https://github.com/bytedance/piano_transcription/blob/master/pytorch/main.py#L230

2. repo只上传了最终的模型。如果改了模型结构，需要重新训练。

3. Model = eval(model_type)，model =

Model(frames_per_second=frames_per_second, classes_num=classes_num)等价于model = Regress_onset_offset_frame_velocity_CRNN(frames_per_second=frames_per_second, classes_num=classes_num)。

4. 例如把

https://github.com/bytedance/piano_transcription/blob/master/pytorch/models.py#L108之后几行的in_channels, out_channels数目减少。例如全部除2。我们没有尝试过，但预计F1值不会低于90%。

5. 训练中不会删除或合并数据。因此可以检查是否误操作了hdf5。

有任何疑问欢迎随时交流！

Best wishes,

Qiuqiang

On Wed, 23 Dec 2020 at 11:15, GUOhm230 notifications@github.com wrote:

作者您好： 非常感谢您再次百忙之中抽出时间帮我答疑解惑，深感惶恐和荣幸。根据最新进度，仍然有些问题想打扰您宝贵的时间帮我答疑解惑。 1.昨天正在尝试做模型训练，由于输出形式和您给出的样板不一样，而且checkpoint文件夹中没有保存每次iteration的pth，不知道我的训练是否是对的，而且我没办法知道到底进行到哪一次了。输出结果如：48075 tensor(0.21980,....)

2.请问作者在代码中（pytorch\main.py）中是否没有加入您已经训练好得模型参数？（CRNN_note_F1=0.9677_pedal_F1=0.9186.pth）我在代码中没有看到。我如果更改了模型结构（比如双向改单向，减少隐层数）还能加载您的模型进行finetune嘛？如果加入的话，是否代码写法如下： Model = eval(model_type) model = Model(frames_per_second=frames_per_second, classes_num=classes_num) load_model = torch.load('./CRNN_note_F1=0.9677_pedal_F1=0.9186.pth') model.load_state_dict(load_model) 即可 3.您代码中的建立模型部分是我不太理解的： Model = eval(model_type) model = Model(frames_per_second=frames_per_second, classes_num=classes_num) 该语句好像并没有调用，pytorch中的models.py中的内容，那是怎么做到的呢？我百度无果，所以请教。 4.您昨天所说的，把所有参数数目减少，比如feature maps,准确率达到90%以上，您是否有尝试.我对于参数数目减少的具体操作不太明白，怎么减少？减少多少？依据是什么呢？

5.之前我在训练时，将MP3文件处理成输入的hdf5文件时是24.9G，也包含了所有数据的MP3文件。而昨天多次训练发现处理后的数据变成了4.1G，比如2004原本有MP3文件132项，之前是132项全部转化成hdf5，而昨天之后只有其中的11项转成了。是否是在训练过程中会把数据删除还是合并？或者只保存训练数据，其中测试数据和评价数据不再保存？

连续叨扰，而且问题繁琐给作者造成诸多不便，希望您可以理解。您的项目实在很有价值，我作为初学者，想把整个项目完整的彻底的捋一遍，每个问题都是自己思考后，百度后方敢请教于您。十分期待您的回复。谢谢 祝事业顺利 GUOhm230

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-749903147, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSMNDXVC52SAIHQNER3SWFOF3ANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 昨晚收到您的回复，感谢您深夜给我这么有质量的回答，基本解决了我的疑问。再次感谢您！目前，仍有如下小问题，尝试解决无果，故再次请教，再次打扰，十分抱歉:

  1. 关于最后一条（数据特征提取出错），发现报错如下： { File "utils/features.py", line 160, in pack_maestro_dataset_to_hdf5(args) File "utils/features.py", line 56, in pack_maestro_dataset_tohdf5 (audio, ) = librosa.core.load(audio_path, sr=sample_rate, mono=True) File "/mnt/softwares/anaconda3/lib/python3.7/site-packages/librosa/core/audio.py", line 126, in load for frame in input_file: File "/mnt/softwares/anaconda3/lib/python3.7/site-packages/audioread/rawread.py", line 136, in read_data data = audioop.lin2lin(data, old_width, TARGET_WIDTH) audioop.error: not a whole number of frames }

    但是奇怪的是，每次都是从头开始，执行到第224个就停止，后面的开始报以上错误。但是我从第225个开始执行，则后面可以执行，但是到某个位置又会出相同的错误。

    为此我把数据重新下载，导入斤对应的文件夹。所有的包按照您文本上的要求安装后，问题依旧。 代码不曾改动，数据也不改动。不知道是不是哪个环境出了问题。还请作者赐教. 2:.最终模型的意思就是，这个模型只能重新开始训练。所有的参数初始值不能以您给出的pth文件内的参数作为初始I值，而只能随机开始或从0开始训练？ 3.300k次迭代，没有epoch嘛？20k次存储一次，意思是一共15轮（epoch）还是仅有一轮，因为在论文，和代码中都没有看到类似的代码数据，因此求教！一个模型300K次迭代，一共有3次这样的过程，一共就是900k次，耗费9天左右。是这个时间估算嘛？ 以上问题，有些繁琐。连续打扰一周，请您海涵。不胜感激 祝身体健康 GuoHM230

qiuqiangkong commented 3 years ago

您好！

1. 看起来是读音频出错了，检查一下librosa版本，我用的是0.6.0。检查一下音频是否损坏。

2. 如果修改了模型结构，就需要重新训练。

3. 300k

iteration对应了大概20个epoch。我一般习惯记录iteration，因为在需要均衡训练样本的情况下epoch没有意义。Note和和Pedal模型各需要300k iteration。合计600k iteration。

Best wishes,

Qiuqiang

On Thu, 24 Dec 2020 at 16:08, GUOhm230 notifications@github.com wrote:

作者您好： 昨晚收到您的回复，感谢您深夜给我这么有质量的回答，基本解决了我的疑问。再次感谢您！目前，仍有如下小问题，尝试解决无果，故再次请教，再次打扰，十分抱歉:

  1. 关于最后一条（数据特征提取出错），发现报错如下： { File "utils/features.py", line 160, in pack_maestro_dataset_to_hdf5(args) File "utils/features.py", line 56, in pack_maestro_dataset_tohdf5 (audio, ) = librosa.core.load(audio_path, sr=sample_rate, mono=True) File "/mnt/softwares/anaconda3/lib/python3.7/site-packages/librosa/core/audio.py", line 126, in load for frame in input_file: File "/mnt/softwares/anaconda3/lib/python3.7/site-packages/audioread/rawread.py", line 136, in read_data data = audioop.lin2lin(data, old_width, TARGET_WIDTH) audioop.error: not a whole number of frames }

但是奇怪的是，每次都是从头开始，执行到第224个就停止，后面的开始报以上错误。但是我从第225个开始执行，则后面可以执行，但是到某个位置又会出相同的错误。

为此我把数据重新下载，导入斤对应的文件夹。所有的包按照您文本上的要求安装后，问题依旧。 代码不曾改动，数据也不改动。不知道是不是哪个环境出了问题。还请作者赐教. 2:.最终模型的意思就是，这个模型只能重新开始训练。所有的参数初始值不能以您给出的pth文件内的参数作为初始I值，而只能随机开始或从0开始训练？

3.300k次迭代，没有epoch嘛？20k次存储一次，意思是一共15轮（epoch）还是仅有一轮，因为在论文，和代码中都没有看到类似的代码数据，因此求教！一个模型300K次迭代，一共有3次这样的过程，一共就是900k次，耗费9天左右。是这个时间估算嘛？ 以上问题，有些繁琐。连续打扰一周，请您海涵。不胜感激 祝身体健康 GuoHM230

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-750798595, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSMG6OX7KTALNYG6T6DSWLZINANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 周末打扰，十分抱歉，祝您周末愉快！ 上次的问题已经排除，问题出在音频文件的问题，编号224和533的文件从csv中剔除后，可以正常训练。目前只能达到训练您给出的模型的地步。自己修改总是出现数据不匹配的问题。应该是修改的参数没有达到统一的原因。因此有如下问题请教于您： 1.我同时将models.py中所有GRU中的bidirection==True改成False。将所有的outchannals缩小为之前的2倍，全链接层从512变成256，Gru的隐藏层同时缩小为原来的一半。请问这样的操作会影响输出维度嘛？。因为我并没有改变输入大小，卷积核大小和填充值这些内容。但是以上的修改多次尝试均会报错,请您赐教！ 2.您在自己构建模型的时候，是否尝试过将模型（pth）缩减至80M左右，且能保证准确率，或者F1值为90%以上呢。 再次打扰，深表歉意 祝您万事如意 GuoHM230

qiuqiangkong commented 3 years ago

您好！

1.

数据不匹配看看报错原因？可能是channels数不匹配。如果可以跑通就表示可以，如果不能跑通就表示维数没有对上。最终的输出总是88。有报错信息吗？

1. 我们没有试过pth缩小到80M左右，但根据我们的经验缩小到80M之后F1应该可以保证在90%以上。

Best wishes,

Qiuqiang

On Sat, 26 Dec 2020 at 21:13, GUOhm230 notifications@github.com wrote:

作者您好： 周末打扰，十分抱歉，祝您周末愉快！

上次的问题已经排除，问题出在音频文件的问题，编号224和533的文件从csv中剔除后，可以正常训练。目前只能达到训练您给出的模型的地步。自己修改总是出现数据不匹配的问题。应该是修改的参数没有达到统一的原因。因此有如下问题请教于您： 1.我同时将models.py http://1.xn--models-vh6j404c0zm5oj.py 中所有GRU中的bidirection==True改成False。将所有的outchannals缩小为之前的2倍，全链接层从512变成256，Gru的隐藏层同时缩小为原来的一半。请问这样的操作会影响输出维度嘛？。因为我并没有改变输入大小，卷积核大小和填充值这些内容。但是以上的修改多次尝试均会报错,请您赐教！ 2.您在自己构建模型的时候，是否尝试过将模型（pth）缩减至80M左右，且能保证准确率，或者F1值为90%以上呢。 再次打扰，深表歉意 祝您万事如意 GuoHM230

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-751354612, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSL2WT7QQUTLJW5AEN3SWXOQXANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 新的一周，祝您生活工作愉快。再次打扰，十分抱歉。因为修改models.py文件中的in_channals和out_channals以及biGRU改成单向后报错如下： size mismatch for frame_model.gru.weight_ih_l1: copying a param with shape torch.Size([768, 512]) from checkpoint, the shape in current model is torch.Size([768, 256]) size mismatch for velocity_model.conv_block4.bn2.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([64]). size mismatch for velocity_model.gru.weight_ih_l1: copying a param with shape torch.Size([768, 512]) from checkpoint, the shape in current model is torch.Size([768, 256]). size mismatch for reg_offset_model.conv_block3.conv2.weight: copying a param with shape torch.Size([96, 96, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 48, 3, 3]). size mismatch for reg_offset_model.conv_block4.conv1.weight: copying a param with shape torch.Size([128, 96, 3, 3]) from checkpoint, the shape in current model is torch.Size([64, 48, 3, 3]). size mismatch for frame_model.conv_block2.conv1.weight: copying a param with shape torch.Size([64, 48, 3, 3]) from checkpoint, the shape in current model is torch.Size([32, 24, 3, 3]). size mismatch for frame_model.conv_block2.conv2.weight: copying a param with shape torch.Size([64, 64, 3, 3]) from checkpoint, the shape in current model is torch.Size([32, 32, 3, 3]). size mismatch for frame_model.conv_block2.bn1.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([32]). size mismatch for frame_model.conv_block2.bn1.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([32]). ， 问题应该是models中的参数虽然修改，但是因为加载了之前的pth模型数据。这里面的参数size还没用修改，所以不匹配报错。 因此应该有以下解决方式： 1.修改pth中的参数size和修改统一 2.不加在pth，设置初始化后开始训练 但是我百度询问多次还是修改不了。所以请教作者。您的pth中的文件是字典型，但是里面每层的参数是怎么取出来并修改的？请您指教。不胜感激 祝好 GuoHM230

GUOhm230 commented 3 years ago

作者您好： 上一个因为网络结构修改导致模型参数不匹配的问题已经按照上面提出得到1设想解决。不知道您看了上个提问嘛（2020.12.28上午的提问）？但是现在又有新的错误如下： Traceback (most recent call last): File "pytorch/inference.py", line 263, in <module> inference(args) File "pytorch/inference.py", line 226, in inference transcribed_dict = transcriptor.transcribe(audio, midi_path) File "pytorch/inference.py", line 92, in transcribe output_dict = forward(self.model, segments, batch_size=1) File "/mnt/omr/GUO/midiTranscriptionTrain/pytorch/pytorch_utils.py", line 107, in forward batch_output_dict = model(batch_waveform) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 161, in forward outputs = self.parallel_apply(replicas, inputs, kwargs) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 171, in parallel_apply return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)]) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 86, in parallel_apply output.reraise() File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/_utils.py", line 428, in reraise raise self.exc_type(msg) RuntimeError: Caught RuntimeError in replica 0 on device 0. Original Traceback (most recent call last): File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker output = module(*input, **kwargs) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "/mnt/omr/GUO/midiTranscriptionTrain/pytorch/models.py", line 351, in forward note_output_dict = self.note_model(input) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "/mnt/omr/GUO/midiTranscriptionTrain/pytorch/models.py", line 233, in forward frame_output = self.frame_model(x) # (batch_size, time_steps, classes_num) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "/mnt/omr/GUO/midiTranscriptionTrain/pytorch/models.py", line 153, in forward output = torch.sigmoid(self.fc(x)) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 93, in forward return F.linear(input, self.weight, self.bias) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/functional.py", line 1692, in linear output = input.matmul(weight.t()) RuntimeError: mat1 dim 1 must match mat2 dim 0 我认为问题应该是我只是将GRU里面的bidirection参数改成False.因此没有做到输入输出的匹配。但是我实在不知道怎么做到统一了。请您有空指教，十分感谢 祝身体健康，万事如意 GuoHM230

qiuqiangkong commented 3 years ago

您好！一般来说，如果改动了模型，则需要重新训练，不能直接读取之前的模型参数。如果要强行读取之前的参数，可以使用model.load_state_dict(strict=False)。但这个操作需要非常谨慎使用。

Best wishes,

Qiuqiang

On Mon, 28 Dec 2020 at 20:24, GUOhm230 notifications@github.com wrote:

作者您好：

上一个因为网络结构修改导致模型参数不匹配的问题已经按照上面提出得到1设想解决。不知道您看了上个提问嘛（2020.12.28上午的提问）？但是现在又有新的错误如下： Traceback (most recent call last): File "pytorch/inference.py", line 263, in inference(args) File "pytorch/inference.py", line 226, in inference transcribed_dict = transcriptor.transcribe(audio, midi_path) File "pytorch/inference.py", line 92, in transcribe output_dict = forward(self.model, segments, batch_size=1) File "/mnt/omr/GUO/midiTranscriptionTrain/pytorch/pytorch_utils.py", line 107, in forward batch_output_dict = model(batch_waveform) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, kwargs) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 161, in forward outputs = self.parallel_apply(replicas, inputs, kwargs) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 171, in parallel_apply return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)]) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 86, in parallel_apply output.reraise() File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/_utils.py", line 428, in reraise raise self.exc_type(msg) RuntimeError: Caught RuntimeError in replica 0 on device 0. Original Traceback (most recent call last): File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker output = module(*input, *kwargs) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(input, kwargs) File "/mnt/omr/GUO/midiTranscriptionTrain/pytorch/models.py", line 351, in forward note_output_dict = self.note_model(input) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, kwargs) File "/mnt/omr/GUO/midiTranscriptionTrain/pytorch/models.py", line 233, in forward frame_output = self.frame_model(x) # (batch_size, time_steps, classes_num) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, *kwargs) File "/mnt/omr/GUO/midiTranscriptionTrain/pytorch/models.py", line 153, in forward output = torch.sigmoid(self.fc(x)) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(input, kwargs) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 93, in forward return F.linear(input, self.weight, self.bias) File "/mnt/softwares/anaconda3/envs/guo/lib/python3.7/site-packages/torch/nn/functional.py", line 1692, in linear output = input.matmul(weight.t()) RuntimeError: mat1 dim 1 must match mat2 dim 0

我认为问题应该是我只是将GRU里面的bidirection参数改成False.因此没有做到输入输出的匹配。但是我实在不知道怎么做到统一了。请您有空指教，十分感谢 祝身体健康，万事如意 GuoHM230

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-751695476, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSOWBGRLQLHSA475Q43SXB2IZANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 请教问题实在有点频繁，还请谅解。 以上的预训练模型的参数不匹配问题已经解决。我通过以下代码修改了下载模型中的torch.size:

import torch
check_point = "./CRNN_note_F1=0.9677_pedal_F1=0.9186.pth"
# check_point = r'E:\Byte_Dance_project\pthModi.pth'
device = torch.device("cuda" if torch.cuda.is_available() else "cup")
check_point = torch.load(check_point, map_location=device)
import collections
dicts = collections.OrderedDict()
model = check_point['model']
note_model = model['pedal_model']
for k, value in note_model.items():
    if k == 'frame_model.gru.weight_ih_l1':
        print(value)
        print(value[1])
        print(value[0][1])
        value = torch.ones(768, 256)
        # print(value)
        # print(type(value))
    if k == 'reg_onset_model.gru.weight_ih_l1':
        print(value)
        value = torch.ones(768, 256)
        print(value)
        print(type(value))
    if k == 'reg_offset_model.gru.weight_ih_l1':
        print(value)
        print(value[1])
        print(value[0][1])
        value = torch.ones(768, 256)
        # print(value)
        # print(type(value))
    if k == 'velocity_model.gru.weight_ih_l1':
        print(value)
        value = torch.ones(768, 256)
        print(value)
        print(type(value))
    if k == 'reg_pedal_onset_model.gru.weight_ih_l1':
        print(value)
        value = torch.ones(768, 256)
        print(value)
        print(type(value))
    if k == 'reg_pedal_offset_model.gru.weight_ih_l1':
        print(value)
        value = torch.ones(768, 256)
        print(value)
        print(type(value))
    if k == 'reg_pedal_frame_model.gru.weight_ih_l1':
        print(value)
        value = torch.ones(768, 256)
        print(value)
        print(type(value))
    note_model[k] = value
torch.save(check_point, "pthModi2.pth")

故应该可以读取之前的参数去学习，且无需model.load_state_dict(strict=False)这个操作。因为可以做到指定数据的裁剪。 但是现在的报错是：RuntimeError: mat1 dim 1 must match mat2 dim 0 我认为问题应该是我只是将GRU里面的bidirection参数改成False.因此没有做到输入输出的匹配。但是我实在不知道怎么做到统一了。请您有空指教，十分感谢

qiuqiangkong commented 3 years ago

这个错误我也没有遇到过，但是看情况好像是形状不对，bidirection改成False之后GRU的输出维度会除2，是不是这个原因？

On Tue, 29 Dec 2020 at 10:16, GUOhm230 notifications@github.com wrote:

作者您好： 请教问题实在有点频繁，还请谅解。 以上的预训练模型的参数不匹配问题已经解决。我通过以下代码修改了下载模型中的torch.size: `import torch check_point = "./CRNN_note_F1=0.9677_pedal_F1=0.9186.pth" check_point = r'E:\Byte_Dance_project\pthModi.pth'

device = torch.device("cuda" if torch.cuda.is_available() else "cup") check_point = torch.load(check_point, map_location=device) import collections dicts = collections.OrderedDict() model = check_point['model'] note_model = model['pedal_model'] for k, value in note_model.items(): if k == 'frame_model.gru.weight_ih_l1': print(value) print(value[1]) print(value[0][1]) value = torch.ones(768, 256)

print(value)

print(type(value))

if k == 'reg_onset_model.gru.weight_ih_l1': print(value) value = torch.ones(768, 256) print(value) print(type(value)) if k == 'reg_offset_model.gru.weight_ih_l1': print(value) print(value[1]) print(value[0][1]) value = torch.ones(768, 256)

print(value)

print(type(value))

if k == 'velocity_model.gru.weight_ih_l1': print(value) value = torch.ones(768, 256) print(value) print(type(value))

if k == 'reg_pedal_onset_model.gru.weight_ih_l1':

print(value)

value = torch.ones(768, 256)

print(value)

print(type(value))

if k == 'reg_pedal_offset_model.gru.weight_ih_l1':

print(value)

value = torch.ones(768, 256)

print(value)

print(type(value))

if k == 'reg_pedal_frame_model.gru.weight_ih_l1':

print(value)

value = torch.ones(768, 256)

print(value)

print(type(value))

note_model[k] = value

torch.save(check_point, "pthModi2.pth")` 故应该可以读取之前的参数去学习，且无需model.load_state_dict(strict=False)这个操作。因为可以做到指定数据的裁剪。

但是现在的报错是：RuntimeError: mat1 dim 1 must match mat2 dim 0 我认为问题应该是我只是将GRU里面的bidirection参数改成False.因此没有做到输入输出的匹配。但是我实在不知道怎么做到统一了。请您有空指教，十分感谢

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-751921849, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSOB4OQG5LCR76WL4RLSXE3XLANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 首先真诚的祝您元旦快乐，新年诸事顺利，身体健康。 我已经完成了RNN网络的修改，并开始训练。但是预计不会降低太多内存，效果未知。所以目前想尝试做通道裁剪降低模型效果，不过方法未知，预后也不清楚。 所以我想请教您， 1.您所认为的内存可以降低到80左右，并保持效果类似或者低几个百分点的依据是什么？您可以给我提供一个这样模型缩减的思路嘛？是直接将GRU减半？将全连接层减半？或者将convBlock的结构缩减？ 2.您在这方面是专家大佬，我才刚入门这个领域，所以请问您之前遇到过钢琴多音符转录的更轻量级模型嘛？如果有论文分享就更好了。十分感激 再次祝好 郭辉铭

qiuqiangkong commented 3 years ago

你好！

1. 根据我们之前做其他任务的经验，比如audio tagging，把全联接层、卷积层、循环层所有参数减半即可。

2. 关于轻量级的模型，可以阅读图像识别领域MobileNet系列文章，可以借鉴到钢琴转谱任务上。

Best wishes,

Qiuqiang

On Thu, 31 Dec 2020 at 10:13, GUOhm230 notifications@github.com wrote:

作者您好： 首先真诚的祝您元旦快乐，新年诸事顺利，身体健康。 我已经完成了RNN网络的修改，并开始训练。但是预计不会降低太多内存，效果未知。所以目前想尝试做通道裁剪降低模型效果，不过方法未知，预后也不清楚。 所以我想请教您，

1.您所认为的内存可以降低到80左右，并保持效果类似或者低几个百分点的依据是什么？您可以给我提供一个这样模型缩减的思路嘛？是直接将GRU减半？将全连接层减半？或者将convBlock的结构缩减？ 2.您在这方面是专家大佬，我才刚入门这个领域，所以请问您之前遇到过钢琴多音符转录的更轻量级模型嘛？如果有论文分享就更好了。十分感激 再次祝好 郭辉铭

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-752817705, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSMFFESXOIBLZI7KFPDSXPM3TANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 感谢您的回复！ 1.您所说的做其他任务是用当前模型转录成midi文件后的任务吗？还是用其他其他模型进行的任务 2.另外，我将RNN和fc的系数减半后，发现loss下降很慢，目前到7万步，从4万步开始就一直保持0.7的loss值。3万多次迭代都没有什么改变。请问这种情况下是已经不需要训练了嘛？

qiuqiangkong commented 3 years ago

您好！

1. 是其他非钢琴转谱的任务，模型参数减半能有效加速计算。

2. 如果不减半训练的loss是多少？loss只是参考指标，最终是以evaluate的结果为准，可以试试inference看F1是多少。

Best wishes,

Qiuqiang

On Thu, 31 Dec 2020 at 16:24, GUOhm230 notifications@github.com wrote:

作者您好： 感谢您的回复！ 1.您所说的做其他任务是用当前模型转录成midi文件后的任务吗？还是用其他其他模型进行的任务

2.另外，我将RNN和fc的系数减半后，发现loss下降很慢，目前到7万步，从4万步开始就一直保持0.7的loss值。3万多次迭代都没有什么改变。请问这种情况下是已经不需要训练了嘛？

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-752886452, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSJHSYRGLA4NKHFCUE3SXQYNHANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 休假期间打扰，十分抱歉。前两天没来实验室，今天才给您回复消息。 1.第一个模型在0.7的时候跑了几万步不动，后面降下来没有就不太清楚了。第二个模型到第十万步，维持0.3的loss也有3万多步了。至于后面会怎么样，还在看结果。 2.目前学习到一些论文，里面有关于通道裁剪和核裁剪的方法。目前还没尝试。不知道能否管用

qiuqiangkong commented 3 years ago

您好！

好的，可以后续观察F1值看看效果如何。

Best wishes,

Qiuqiang

On Sun, 3 Jan 2021 at 15:42, GUOhm230 notifications@github.com wrote:

作者您好： 休假期间打扰，十分抱歉。前两天没来实验室，今天才给您回复消息。

1.第一个模型在0.7的时候跑了几万步不动，后面降下来没有就不太清楚了。第二个模型到第十万步，维持0.3的loss也有3万多步了。至于后面会怎么样，还在看结果。 2.目前学习到一些论文，里面有关于通道裁剪和核裁剪的方法。目前还没尝试。不知道能否管用

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-753580828, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSN4RGKJXWE7YV6HE73SYANYBANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 再次打扰您，请教一个对您来说简单的问题。 1.请问您所说的将全链接层的减半是指对linear函数减半吧？将循环层减半是指在GRU函数中将hidden layer减半并将双向改成多向对吗？将卷积层减半是指对in_channal和out_channal数据减半对吗？ 2.您的代码中怎么才能做到不添加您训练得到的pth模型中的数据，而重新开始训练？请您指教

qiuqiangkong commented 3 years ago

您好！

1. 是的。

2. 只要把load模型相关的部分去掉，从头开始训练即可，即使用main.py train。

Best wishes,

Qiuqiang

On Sun, 3 Jan 2021 at 19:06, GUOhm230 notifications@github.com wrote:

作者您好： 再次打扰您，请教一个对您来说简单的问题。 1.请问您所说的将全链接层的减半是指对linear函数减半吧？将循环层减半是指在GRU函数中将hidden layer减半并将双向改成多向对吗？将卷积层减半是指对in_channal和out_channal数据减半对吗？ 2.您的代码中怎么才能做到不添加您训练得到的pth模型中的数据，而重新开始训练？请您指教

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-753600364, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSMTYFVRNK7IDODIIQ3SYBFSLANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 1.main.py中的230行开始是每20000次迭代保存一次模型，300000次迭代，共保存15次模型。那么一个note,一个pedal在checkpoint文件夹中就保存了30个pth模型。但是在组合两个模型为最终的模型时，我并没有看到怎么把这15个综合起来的代码？所以请问您，在Combine the note and pedal models阶段，怎么只是传入Regress_onset_offset_frame_velocity_CRNN_onset_F1=0.9677.pth和Regress_pedal_CRNN_onset_F1=0.9186.pth？而这两个文件是在哪生成的？ 2.我看到在训练过程中，如果传入您给的pth中的参数。大概1万步以后模型就不再变动。因此我在训练的时候，能否降低至10万步？但是在重新训练的过程中仍然训练30万次？

qiuqiangkong commented 3 years ago

您好！

1. 合并note和pedal模型看这里：

https://github.com/bytedance/piano_transcription/blob/master/runme.sh#L26 。需要先独立训练note和pedal模型才能合并。

2. 在训练中使用已有pth做初始化？如果是自己的模型不需要load模型初始化，直接随机初始化即可。10万步或者30万步都可以。

Best wishes, Qiuqiang

On Tue, 5 Jan 2021 at 11:09, GUOhm230 notifications@github.com wrote:

作者您好： 1.main.py中的230行开始是每20000次迭代保存一次模型，300000次迭代，共保存15次模型。那么一个note,一个pedal在checkpoint文件夹中就保存了30个pth模型。但是在组合两个模型为最终的模型时，我并没有看到怎么把这15个综合起来的代码？所以请问您，在Combine the note and pedal models阶段，怎么只是传入Regress_onset_offset_frame_velocity_CRNN_onset_F1=0.9677.pth和Regress_pedal_CRNN_onset_F1=0.9186.pth？而这两个文件是在哪生成的？

2.我看到在训练过程中，如果传入您给的pth中的参数。大概1万步以后模型就不再变动。因此我在训练的时候，能否降低至10万步？但是在重新训练的过程中仍然训练30万次？

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-754360353, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSMWGGUTP2UZ5YLIPTDSYJ7F5ANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

您好： 感谢您百忙之中的回复，祝工作顺利。 1.合并模型我大概知道意思，但是我不明白的是，迭代之后生成很多的pth,而合并是需要note训练的一个和padal训练中的一个。因此我不明白这个pth文件在哪个位置？代码中也没有看到这个pth生成以及储存的地方。

  1. 我改变了您提供的pth文件的结构和对应的model结构参数，使得inference同样可以运行成功并转录。但是转录效果很差，或者说不U能转录，请问这种情况是为什么呢？ 祝好 郭辉铭
qiuqiangkong commented 3 years ago

您好！

1.

迭代生成很多pth选最好（最后）的，note和pedal是分别训练两个模型产生的pth，需要手动把他们找出来，通常在checkpoint文件夹下。

2. 那说明模型没训练好。跑一下最初的未修改的系统的训练，inference一下就知道是否成功了。

Best wishes,

Qiuqiang

On Tue, 5 Jan 2021 at 11:56, GUOhm230 notifications@github.com wrote:

您好： 感谢您百忙之中的回复，祝工作顺利。

1.合并模型我大概知道意思，但是我不明白的是，迭代之后生成很多的pth,而合并是需要note训练的一个和padal训练中的一个。因此我不明白这个pth文件在哪个位置？代码中也没有看到这个pth生成以及储存的地方。 2. 我改变了您提供的pth文件的结构和对应的model结构参数，使得inference同样可以运行成功并转录。但是转录效果很差，或者说不U能转录，请问这种情况是为什么呢？ 祝好 郭辉铭

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-754375794, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSIXGJXE2NRCULA6FCLSYKEVNANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 感谢您的回复，目前方向比较明确。相比之前的暴力裁剪，我觉得最近看的一篇模型裁剪的论文可能更有作用。正在尝试，如果能做到模型大幅度的削减并保证精度在一个合理的范围，我会向您汇报。感谢您的支持，理解以及教导。 1.您的GiantMIDI-Piano 数据集内包含大量的经典钢琴曲，请问经过转录后是否经过专业校对呢？ 2.请求您可以将GiantMIDI-Piano 数据集数据集发送一份到我的邮件嘛？我尝试试试通过这些文件做一个矢量化的东西。十分感谢！邮箱：iguohm@163.com.(您的该处github上说通过邮件获得您发送的下载资格和链接) 3.另外，由于您是这个领域得到前辈和专家，而我只是刚入门的学生，所以请问您是否知道目前有没有除了深度学习以外的方法可以做到一个音频音符和已知音符之间是否可以匹配。也就是说，没有两个完全一样的音符，理论来说是肯定存在某些特征可以比较完美的区分两个音符。那么您有这方面的资料或者建议嘛？ 又是一些超出合理界限的问题，还请您原谅，如果您事业繁忙，只需要获得数据集下载链接亦可。但是学生真诚希望获得更多有用的信息。十分感谢 真诚的祝您身体健康，万事如意 郭辉铭

qiuqiangkong commented 3 years ago

您好！

1. 我们在50首曲子上进行了校准测验，详细可参考：https://arxiv.org/abs/2010.07061

2. GiantMIDI-Piano下载链接请见邮件底部。

3. 关于对齐问题，可以参看“PERFORMANCE ERROR DETECTION AND POST-PROCESSING FOR FAST

AND ACCURATE SYMBOLIC MUSIC ALIGNMENT”一文。

Best wishes,

Qiuqiang

===================== Hi,

Thanks for your email! Please download GiantMIDI-Piano here:

Google drive: https://drive.google.com/drive/folders/1Stz3CAvMoplo79LR5I3onMWRelCugBYS?usp=sharing

Or: 百度网盘：https://pan.baidu.com/s/1up4jzPcalVMJt5RfYhEerg 密码: gbl0

It is OK to forward this email and dataset to others as long as the

disclaimer is remained.

Best wishes,

Qiuqiang Kong Research Scientist Speech, Audio and Music Intelligence (SAMI) ByteDance

Disclaimer Your access to and use of this dataset are at your own risk. We do not guarantee the accuracy of this dataset. The dataset is provided “as is” and we make no warranty or representation to you with respect to it and we expressly disclaim, and hereby expressly waive, all warranties, express, implied, statutory or otherwise. This including, without limitation, warranties of quality, performance, merchantability or fitness for a particular purpose, non-infringement, absence of latent or other defects, accuracy, or the presence or absence of errors, whether or not known or discoverable. Where disclaimers of warranties are not allowed in full or in part, this disclaimer may not apply to you.

In no event will we be liable to you on any legal theory (including, without limitation, negligence) or otherwise for any direct, special, indirect, incidental, consequential, punitive, exemplary, or other losses, costs, expenses, or damages arising out of this public license or use of the licensed material.

The disclaimer of warranties and limitation of liability provided above shall be interpreted in a manner that, to the extent possible, most closely approximates an absolute disclaimer and waiver of all liability.

On Wed, 6 Jan 2021 at 09:58, GUOhm230 notifications@github.com wrote:

作者您好：

感谢您的回复，目前方向比较明确。相比之前的暴力裁剪，我觉得最近看的一篇模型裁剪的论文可能更有作用。正在尝试，如果能做到模型大幅度的削减并保证精度在一个合理的范围，我会向您汇报。感谢您的支持，理解以及教导。 1.您的GiantMIDI-Piano 数据集内包含大量的经典钢琴曲，请问经过转录后是否经过专业校对呢？ 2.请求您可以将GiantMIDI-Piano 数据集数据集发送一份到我的邮件嘛？我尝试试试通过这些文件做一个矢量化的东西。十分感谢！邮箱： iguohm@163.com.(您的该处github上说通过邮件获得您发送的下载资格和链接)

3.另外，由于您是这个领域得到前辈和专家，而我只是刚入门的学生，所以请问您是否知道目前有没有除了深度学习以外的方法可以做到一个音频音符和已知音符之间是否可以匹配。也就是说，没有两个完全一样的音符，理论来说是肯定存在某些特征可以比较完美的区分两个音符。那么您有这方面的资料或者建议嘛？ 又是一些超出合理界限的问题，还请您原谅，如果您事业繁忙，只需要获得数据集下载链接亦可。但是学生真诚希望获得更多有用的信息。十分感谢 真诚的祝您身体健康，万事如意 郭辉铭

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-755025925, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSJWJYOTG2FPVPLCLNTSYO7U7ANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 感谢您的回复，我已经按照您的链接下载了midi文件。在您百忙之中再次打扰，十分抱歉。目前对于您模型的修改，我有以下疑惑： 1.我修改了gru从双向变成单向之后，每个模型训练10万次迭代后进行测试，发现转录之后的MIDI文件能识别出很多音符，准确率自然就很差请问是训练次数太少？还是因为没了双向的约束而导致后来的每一帧都成了音符判定帧了的问题？您在构建模型的时候，有过这些思考嘛？

qiuqiangkong commented 3 years ago

您好！

1. 我没有测试过双向网络，猜测双向网络会使效果变差，但也不会差很多。

Best wishes,

Qiuqiang

On Fri, 8 Jan 2021 at 10:34, GUOhm230 notifications@github.com wrote:

作者您好： 感谢您的回复，我已经按照您的链接下载了midi文件。在您百忙之中再次打扰，十分抱歉。目前对于您模型的修改，我有以下疑惑：

1.我修改了gru从双向变成单向之后，每个模型训练10万次迭代后进行测试，发现转录之后的MIDI文件能识别出很多音符，准确率自然就很差请问是训练次数太少？还是因为没了双向的约束而导致后来的每一帧都成了音符判定帧了的问题？您在构建模型的时候，有过这些思考嘛？

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-756508135, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSIZH22KRH2EPTTOAGDSYZVNFANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

您好： 祝您的周末阳光灿烂。再次冒昧打扰，敬请原谅 1.目前我实验的从双向变成单向网络的结果很差，20万次的迭代之后，转录效果大打折扣，基本无法实现准确的转录。不知道是什么原因。请问您有什么想法意见可以告知嘛？ 2.如果我不实现单双向的转录，而只是把网络层数，卷积通道进行削减，您认为是否可行？ 十分期待您的回复，万分感谢 郭辉铭

qiuqiangkong commented 3 years ago

您好，

1. debug时控制变量，首先跑原始的代码，没问题再改双向成单项GRU。再看看。

2. 削减层数、通道数可行。

Best wishes,

Qiuqiang

On Sun, 10 Jan 2021 at 14:52, GUOhm230 notifications@github.com wrote:

您好： 祝您的周末阳光灿烂。再次冒昧打扰，敬请原谅

1.目前我实验的从双向变成单向网络的结果很差，20万次的迭代之后，转录效果大打折扣，基本无法实现准确的转录。不知道是什么原因。请问您有什么想法意见可以告知嘛？ 2.如果我不实现单双向的转录，而只是把网络层数，卷积通道进行削减，您认为是否可行？ 十分期待您的回复，万分感谢 郭辉铭

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-757427459, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSPQVWH443OJ72QDUULSZFFEBANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 十分感谢您的回答。 1.目前GRU双向改单向已经实现并且训练完毕，效果欠佳，识别出很多无关音符！ 2.您说的这个我还没尝试。下次试试之后有结果再向您请教。 3.请问您的pth模型转换成onnx文件的时候，是怎么设置张量大小的？（torch,randn( )）.因为别人的模型里面张量大小比较统一，而您这个里面的张量比较复杂不一所以想知道这种情况下怎么处理？

qiuqiangkong commented 3 years ago

您好！

1. 原始双向重新训练是否有问题？

3. 我们还没有试过转onnx。

Best wishes,

Qiuqiang

On Tue, 12 Jan 2021 at 17:34, GUOhm230 notifications@github.com wrote:

作者您好： 十分感谢您的回答。 1.目前GRU双向改单向已经实现并且训练完毕，效果欠佳，识别出很多无关音符！ 2.您说的这个我还没尝试。下次试试之后有结果再向您请教。 3.请问您的pth模型转换成onnx文件的时候，是怎么设置张量大小的？（torch,randn( )）.因为别人的模型里面张量大小比较统一，而您这个里面的张量比较复杂不一所以想知道这种情况下怎么处理？

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-758529876, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSNCBM5R4764674URLDSZQJRJANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 1.十分感谢您的提醒，我昨晚用原始的模型，未做任何改动进行训练，发现结果和去除单向时的结果是一致的。请问这是为什么？用的您这上面下载的模型，官网下载的数据，未做改动的训练。 祝生活幸福 郭辉铭

qiuqiangkong commented 3 years ago

您好！

一致的是work还是不work？

Best wishes,

Qiuqiang

On Wed, 13 Jan 2021 at 10:58, GUOhm230 notifications@github.com wrote:

作者您好：

1.十分感谢您的提醒，我昨晚用原始的模型，未做任何改动进行训练，发现结果和去除单向时的结果是一致的。请问这是为什么？用的您这上面下载的模型，官网下载的数据，未做改动的训练。 祝生活幸福 郭辉铭

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bytedance/piano_transcription/issues/1#issuecomment-759171440, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFXTSKYDULPGRE7EE5EHHTSZUD6DANCNFSM4UH6LPDA .

GUOhm230 commented 3 years ago

作者您好： 感激于您百忙之中再次抽空回复我的问题： 1.work,且转录结果一致：用您这里提供的网络结构（包括结构，参数全部是您这上面设定的值）和官网下载的maestrov2.0的数据进行训练，得到的pth模型进行inference，转录结果和我将双向网络换成单向网络（其他参数不变，改的只是单向和对应的全连接层数）后训练出的pth模型结果一致：均能训练出很多无关音符！ 2.请问以上问题是否是因为您上传的网络模型并不是得到这个模型："https://zenodo.org/record/4034264/files/CRNN_note_F1%3D0.9677_pedal_F1%3D0.9186.pth?download=1"的网络结构和参数？ 3.请问您在此做了些什么修改？我可以按照您提供的方法训练出和上面链接一样的pth参数嘛？如果方便的话，您是否可以提供您得到最佳pth时候的网络结构和参数？ 4.我无法将您的pth模型转换成onnx。出错原因还是在数据上，您的网络参数最高是五维而torch.onnx.export()目前不支持 以上问题真诚的等待您打的回复，尤其迫切的希望可以训练出和您一样准确的pth文件。十分感激 祝身体健康 郭辉铭