search

w-okada / voice-changer

リアルタイムボイスチェンジャー Realtime Voice Changer
Other
16.4k stars 1.8k forks source link

[9604] Failed to launch: Can't load ONNX/ [Voice Changer] Launch Exception, [WinError 2] The system cannot find the file s pecified #576

Closed te-ch closed 1 year ago

te-ch commented 1 year ago

Issue Type

Bug Report, Question

vc client version number

MMVCServerSIO_win_onnxgpu-cuda_v.1.5.3.10a

OS

Windows 10

GPU

GTX 1070

Clear setting

yes

Sample model

no

Input chunk num

no

Wait for a while

yes

read tutorial

yes

Extract files to a new folder.

yes

Voice Changer type

none yet

Model type

pytorch cuda

Situation

I am unable to start the app always due to problems with onnxruntime. I have tried both the start_http.bat method and dev method with node and conda. I have tried on every windows version and I get the same error. On the server version with npm start I am able to get to the webui but I get a different error. Using windows command prompt. Japanese text appears as ??? as I changed the system locale to English, as apparently not having it on English causes some problems with ONNX.

From start_http.bat: 

(vcclient-dev) c:/users/name/projects/MMVCServerSIO_win_onnxdirectML-cuda_v.1.5.3.10a\MMVCServerSIO>MMVCServerSIO.exe -p 18888     --https true     --content_vec_500 pretrain/checkpoint_best_legacy_500.pt     --content_vec_500_onnx pretrain/content_vec_500.onnx     --content_vec_500_onnx_on true     --hubert_base pretrain/hubert_base.pt     --hubert_base_jp pretrain/rinna_hubert_base_jp.pt
 --hubert_soft pretrain/hubert/hubert-soft-0d54a1f4.pt     --nsf_hifigan pretrain/nsf_hifigan/model     --crepe_onnx_full pretrain/crepe_onnx_full.onnx     --crepe_onnx_tiny pretrain/crepe_onnx_tiny.onnx     --rmvpe pretrain/rmvpe.pt     --model_dir model_dir     --samples samples.json
Traceback (most recent call last):
  File "MMVCServerSIO.py", line 10, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 385, in exec_module
  File "downloader\SampleDownloader.py", line 9, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 385, in exec_module
  File "voice_changer\DiffusionSVC\DiffusionSVCModelSlotGenerator.py", line 4, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 385, in exec_module
  File "onnxruntime\__init__.py", line 55, in <module>
  File "onnxruntime\__init__.py", line 23, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 385, in exec_module
  File "onnxruntime\capi\_pybind_state.py", line 33, in <module>
ImportError: DLL load failed while importing onnxruntime_pybind11_state: The specified module could not be found.
[16396] Failed to execute script 'MMVCServerSIO' due to unhandled exception!

When I try from inside the voice-changer git repo according to the dev guide : 

(vcclient-dev) C:\users\name\projects\voice-changer\server>python MMVCServerSIO.py -p 18888 --https false --content_vec_500 pretrain/checkpoint_best_legacy_500.pt --content_vec_500_onnx pretrain/content_vec_500.onnx --content_vec_500_onnx_on true --hubert_base pretrain/hubert_base.pt --hubert_base_jp pretrain/rinna_hubert_base_jp.pt --hubert_soft pretrain/hubert/hubert-soft-0d54a1f4.pt --nsf_hifigan pretrain/nsf_hifigan/model --crepe_onnx_full pretrain/crepe_onnx_full.onnx --crepe_onnx_tiny pretrain/crepe_onnx_tiny.onnx --rmvpe pretrain/rmvpe.pt --model_dir model_dir --samples samples.json
  warnings.warn(
    Booting PHASE :__main__
    PYTHON:3.10.12 | packaged by Anaconda, Inc. | (main, Jul  5 2023, 19:01:18)
[MSC v.1916 64 bit (AMD64)]
    Voice Changerを起動しています。
[Voice Changer] download sample catalog. samples_0003_t2.json
[Voice Changer] download sample catalog. samples_0003_o2.json
[Voice Changer] download sample catalog. samples_0003_d2.json
[Voice Changer] model_dir is already exists. skip download samples.
    Internal_Port:18888
    protocol: HTTP
    -- ---- --
    ブラウザで次のURLを開いてください.
    http://<IP>:<PORT>/
    多くの場合は次のいずれかのURLにアクセスすると起動します。
    http://localhost:18888/
[Voice Changer] Launch Exception, [WinError 2] The system cannot find the file specified
  warnings.warn(
    Booting PHASE :__mp_main__
    サーバプロセスを起動しています
  warnings.warn(
    Booting PHASE :__mp_main__
    サーバプロセスを起動しています。
    Booting PHASE :MMVCServerSIO```

Then when loading the UI on localhost:18888/ it will ask for microphone access then comes up with this error 
```Looks like there's a bit of a problem.
unknwon message

Error
unhandledrejection
no error stack

TypeError: n.setSinkId is not a function
yV/</e/</<@http://127.0.0.1:18888/index.js:2:3051815
yV/</e/<@http://127.0.0.1:18888/index.js:2:3051685
h@http://127.0.0.1:18888/index.js:2:1380135
61/r/z/<@http://127.0.0.1:18888/index.js:2:1381491
61/r/w/</<@http://127.0.0.1:18888/index.js:2:1380498
e@http://127.0.0.1:18888/index.js:2:1387428
o@http://127.0.0.1:18888/index.js:2:1387632

I also tried deleting the model_dir and pretrain folders and stored_settings.json but it has come up with the same error. I would really like to run this project and I have tried everything and I appreciate the hard work that went into it. Any help would be greatly appreciated.

w-okada commented 1 year ago

I have tried both the start_http.bat method and dev method with node and conda. I have tried on every windows version

Thank you for trying various situation, but if in all situation this error occurs, it is a very difficult problem. 　Have you included any prerequisite software that is common to each situation?

te-ch commented 1 year ago

thanks for the reply. I don't know, what software would I normally need? I installed pytorch in the conda env and I have visual studio 2019

w-okada commented 1 year ago

Ah, please use chrome.

te-ch commented 1 year ago

I just saw this comment and tried it and it loads thank you! However now I am getting another problem. T-T I forgot to mention before that when I load it I get the error that avresample-4.dll when it is loading (x2) and when I select the model, api-ms-win-core-libraryloader-l1-2-0.dll is missing when I start it but the UI still loads so I don't know if that's relevant. On chrome the UI loads at least now which is amazing!! But when I click start I get pipeline is not initialized. I thought I was saved for a moment, but sadly now there is another ONNX error. 

[Voice Changer] [RVC] Creating instance
VoiceChanger Initialized (GPU_NUM(cuda):1, mps_enabled:False, onnx_device:GPU)
[Voice Changer][RVC]: update_settings gpu:0
[Voice Changer][RVC] Initializing...
2 0 2 3 - 0 7 - 2 9   1 8 : 2 6 : 0 2 . 6 2 7 2 3 8 8   [E : o n n x r u n t i m e : D e f a u l t.   p r o v i d e r _ b r i d g e _ o r t . c c : 1 2 6 6 o n n x r u n t i m e : : T r y G e t P r o v i d e r I n f o _ C U D A ]   D : \ a \ _ w o r k \ l \ s \ o n n x r u n t i m e \ c o r e \ s e s s i o n \ p r o v i d e r L i b r a r y : : G e t   [ O N N X R u n t i m e E r r o r ]   : 1   : F A I L   :   L o a d L i b r a r y   f a i l e d   w i t h   e r r o r   1 2 6   " "   w h e n    t r y i n g   t o   l o a d   " C : \ U s e r s \ n a m e \ m i n i c o n d a 3 \ e n v s \ v c c l i e n t - d e v \ l i b \ s i t e - p a c k a g e s \ o n n x r u n t i m e \ c a p i \ o n n x r u n t i m e _ p r o v i d e r s _ c u d a . d l l "

(yes it actually printed like that)

[Voice Changer]exception! loading inferencer D:\a\_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:574 nnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasn't able to be loaded. Please install the correct version of CUDA and cuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/referenceexecution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.

I have pytorch and cuda and cuddn installed, and when I go into the python terminal on that conda env and type import torch; torch.cuda.is_available() it returns true. (even better, the url provided in that onnx error leads to a 404 haha)

In the console it says [Voice Changer] Pipeline is not initalized.111 over and over when I click start. Frequent errors occur. Please check if the model of the framework being targeted is loaded.

Any ideas from here?

w-okada commented 1 year ago

I have pytorch and cuda and cuddn installed, and when I go into the python terminal on that conda env and type import torch; torch.cuda.is_available() it returns true. (even better, the url provided in that onnx error leads to a 404 haha)

But, app cannot find them. check CUDA_PATH.

te-ch commented 1 year ago

ok CUDA_PATH is this C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7 I also have CUDA_PATH_V10_1 and CUDA_PATH_V11_7 set to the same thing. I already did conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia do I need to 

conda install -c anaconda cudatoolkit
conda install -c anaconda cudnn

or something like this? or is it literally reinstall the latest cudnn on my system since I have 11.7? I am scared to mess up my entire environment and start again from scratch, since I have stable diffusion working and was able to train voice models with this before.

w-okada commented 1 year ago

That's true. It is scary to mess with one's environment. However, I suspect this is an issue with your specific environment, as many other users don't seem to have any problems. In any case, I can't provide detailed advice for individual environments, so I'm at a loss.

te-ch commented 1 year ago

https://github.com/microsoft/onnxruntime/issues/11037#issuecomment-1081722659 I found this, apparently. I don't know how to compile from source but might help others. so far it runs with errors but has very glitchy choppy and inconsistent audio probably on account of only onnx not finding cuda

te-ch commented 1 year ago

installed the latest cudnn and cuda 11.8 and also added CUDNN environment variable and set CUDA_PATH to the latest one, now it works on cpu (nice!) although it's very slow, and when I switch to any gpu model or sometimes even not, I get this error now

[Voice Changer][RVC]: update_settings serverAudioStated:0
[Voice Changer] update configuration: enableServerAudio 0
[Voice Changer][RVC]: update_settings enableServerAudio:0
[Voice Changer] update configuration: f0Detector rmvpe
[Voice Changer][RVC]: update_settings f0Detector:rmvpe
[Voice Changer] post_update_settings ex: cuDNN version incompatibility: PyTorchwas compiled  against (8, 7, 0) but found runtime version (8, 6, 0). PyTorch already comes bundled with cuDNN. One option to resolving this error is to ensure PyTorch can find the bundled cuDNN.
Traceback (most recent call last):
  File "C:\Users\name\Projects\voice-changer\server\restapi\MMVC_Rest_Fileuploader.py", line 71, in post_update_settings
    info = self.voiceChangerManager.update_settings(key, val)
  File "C:\Users\name\Projects\voice-changer\server\voice_changer\VoiceChangerManager.py", line 288, in update_settings
    self.voiceChanger.update_settings(key, val)
  File "C:\Users\name\Projects\voice-changer\server\voice_changer\VoiceChangerV2.py", line 148, in update_settings
    ret = self.voiceChanger.update_settings(key, val)
  File "C:\Users\name\Projects\voice-changer\server\voice_changer\RVC\RVCr2.py", line 79, in update_settings
    pitchExtractor = PitchExtractorManager.getPitchExtractor(self.settings.f0Detector, self.settings.gpu)
  File "C:\Users\name\Projects\voice-changer\server\voice_changer\RVC\pitchExtractor\PitchExtractorManager.py", line 24, in getPitchExtractor
    cls.currentPitchExtractor = cls.loadPitchExtractor(pitchExtractorType,  gpu)

  File "C:\Users\name\Projects\voice-changer\server\voice_changer\RVC\pitchExtractor\PitchExtractorManager.py", line 42, in loadPitchExtractor
    return RMVPEPitchExtractor(cls.params.rmvpe, gpu)
  File "C:\Users\name\Projects\voice-changer\server\voice_changer\RVC\pitchExtractor\RMVPEPitchExtractor.py", line 21, in __init__
    self.rmvpe = RMVPE(model_path=file, is_half=False, device=self.device)
  File "C:\Users\name\Projects\voice-changer\server\voice_changer\DiffusionSVC\p
itchExtractor\rmvpe\rmvpe.py", line 353, in __init__
    self.model = self.model.to(device)
  File "C:\Users\name\miniconda3\envs\vcreal\lib\site-packages\torch\nn\modules\module.py", line 1145, in to
    return self._apply(convert)
  File "C:\Users\name\miniconda3\envs\vcreal\lib\site-packages\torch\nn\modules\module.py", line 797, in _apply
    module._apply(fn)
  File "C:\Users\name\miniconda3\envs\vcreal\lib\site-packages\torch\nn\modules\
module.py", line 797, in _apply
    module._apply(fn)
  File "C:\Users\name\miniconda3\envs\vcreal\lib\site-packages\torch\nn\modules\module.py", line 797, in _apply
    module._apply(fn)
  File "C:\Users\name\miniconda3\envs\vcreal\lib\site-packages\torch\nn\modules\rnn.py", line 202, in _apply
    self._init_flat_weights()
  File "C:\Users\name\miniconda3\envs\vcreal\lib\site-packages\torch\nn\modules\rnn.py", line 139, in _init_flat_weights
    self.flatten_parameters()
  File "C:\Users\name\miniconda3\envs\vcreal\lib\site-packages\torch\nn\modules\rnn.py", line 169, in flatten_parameters
    not torch.backends.cudnn.is_acceptable(fw.data)):
  File "C:\Users\name\miniconda3\envs\vcreal\lib\site-packages\torch\backends\cudnn\__init__.py", line 97, in is_acceptable
    if not _init():
  File "C:\Users\name\miniconda3\envs\vcreal\lib\site-packages\torch\backends\cudnn\__init__.py", line 60, in _init
    raise RuntimeError(base_error_msg)
RuntimeError: cuDNN version incompatibility: PyTorch was compiled  against (8, 7, 0) but found runtime version (8, 6, 0). PyTorch already comes bundled with cuDNN. One option to resolving this error is to ensure PyTorch can find the bundled cuDNN.

hmmmm also this is with a fresh conda env still can't get it to detect with onnx

w-okada commented 1 year ago

Error message said your cudnn is old. I cannot advise you on whether to update or not.

te-ch commented 1 year ago

decided to give this another shot after a few days after being very confused by conflicting documentation and supposed drama with pytorch binaries which may have been the cause of this. I tried literally everything and eventually pinpointed the fact that cuda was being used but only with version 8.6, and not 8.7 which it was compiled against. My eventual solution was:

w-okada commented 1 year ago

Okay, Thanks for your greate reporting.

harumaki4649 commented 1 year ago

私の場合はnvidiaのcuda toolをアンインストールすることによって解決しました。 しかしAIの開発に支障が出るためこの問題を可能であれば修正していただけるとありがたいです