search

w-okada / voice-changer

リアルタイムボイスチェンジャー Realtime Voice Changer
Other
15.34k stars 1.65k forks source link

[ISSUE]: Source code is incomplete or not up to date #1064

Open deiteris opened 6 months ago

deiteris commented 6 months ago

Voice Changer Version

MMVCServerSIO_win_onnxdirectML-cuda_v.1.5.3.17b.zip

Operational System

Windows 10 22H2

GPU

AMD Radeon RX 6600M

Read carefully and check the options

Model Type

RVC

Issue Description

The source code that is currently in the repository is missing the code related to ONNX model embedder and seems to only partially implement DirectML. For example, this is not implemented: https://github.com/w-okada/voice-changer/blob/master/server/voice_changer/RVC/embedder/OnnxContentvec.py.

This is the output of the latest git commit (87d23828286a59c64eceb21042ac53aca5b261a7). You can notice that it falls back to FairseqHubert embedder:

[Voice Changer] [RVCr2] Creating instance
VoiceChangerV2 Initialized (GPU_NUM(cuda):0, mps_enabled:False, onnx_device:CPU-DML)
[Voice Changer][RVC]: update_settings gpu:0
[Voice Changer][RVCr2] Initializing...
2024-01-05 00:43:47.4502837 [W:onnxruntime:, session_state.cc:1162 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-01-05 00:43:47.4556006 [W:onnxruntime:, session_state.cc:1164 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
[Voice Changer] generate new embedder. (no embedder)
[Voice Changer] use torch contentvec Not implemented
[Voice Changer] Loading index...
[Voice Changer] Index file is not found
GENERATE INFERENCER<voice_changer.RVC.inferencer.OnnxRVCInferencer.OnnxRVCInferencer object at 0x000001D5CF40B790>
GENERATE EMBEDDER<voice_changer.RVC.embedder.FairseqHubert.FairseqHubert object at 0x000001D5CF470100>
GENERATE PITCH EXTRACTOR<voice_changer.RVC.pitchExtractor.RMVPEOnnxPitchExtractor.RMVPEOnnxPitchExtractor object at 0x000001D5D24B40D0>

However, ONNX version published on Hugging Face works correctly and uses ONNX embedder. This is the output from the latest release version:

[Voice Changer] [RVCr2] Creating instance
VoiceChangerV2 Initialized (GPU_NUM(cuda):0, mps_enabled:False, onnx_device:CPU-DML)
[Voice Changer][RVC]: update_settings gpu:0
[Voice Changer][RVCr2] Initializing...
2024-01-05 00:12:06.3732371 [W:onnxruntime:, session_state.cc:1169 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-01-05 00:12:06.3787072 [W:onnxruntime:, session_state.cc:1171 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
[Voice Changer] generate new embedder. (no embedder)
[Voice Changer] Loading index...
[Voice Changer] Index file is not found
GENERATE INFERENCER<voice_changer.RVC.inferencer.OnnxRVCInferencer.OnnxRVCInferencer object at 0x000001CD4AEEBDF0>
GENERATE EMBEDDER<voice_changer.RVC.embedder.OnnxContentvec.OnnxContentvec object at 0x000001CD4AEEB550>
GENERATE PITCH EXTRACTOR<voice_changer.RVC.pitchExtractor.RMVPEOnnxPitchExtractor.RMVPEOnnxPitchExtractor object at 0x000001CD4AEEBE50>

Application Screenshot

No response

Logs on console

MMVCServerSIO.exe     -p 18888     --https false     --content_vec_500 pretrain/checkpoint_best_legacy_500.pt     --content_vec_500_onnx pretrain/content_vec_500.onnx     --content_vec_500_onnx_on true     --hubert_base pretrain/hubert_base.pt     --hubert_base_jp pretrain/rinna_hubert_base_jp.pt     --hubert_soft pretrain/hubert/hubert-soft-0d54a1f4.pt     --nsf_hifigan pretrain/nsf_hifigan/model     --crepe_onnx_full pretrain/crepe_onnx_full.onnx     --crepe_onnx_tiny pretrain/crepe_onnx_tiny.onnx     --rmvpe pretrain/rmvpe.pt     --model_dir model_dir     --samples samples.json
    Booting PHASE :__main__
    PYTHON:3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
    Activating the Voice Changer.
[Voice Changer] download sample catalog. samples_0004_t.json
[Voice Changer] download sample catalog. samples_0004_o.json
[Voice Changer] download sample catalog. samples_0004_d.json
[Voice Changer] model_dir is already exists. skip download samples.
    Internal_Port:18888
    protocol: HTTP
    -- ---- --
    Please open the following URL in your browser.
    http://<IP>:<PORT>/
    In many cases, it will launch when you access any of the following URLs.
    http://127.0.0.1:18888/

[VCClient] Access http://127.0.0.1:18888/
[VCClient] wait web server...0 http://127.0.0.1:18888/
    Booting PHASE :__main__
    Booting PHASE :MMVCServerSIO
[Voice Changer] VoiceChangerManager initializing...
[Voice Changer] model slot is changed -1 -> 22
................RVC
[Voice Changer] [RVCr2] Creating instance
VoiceChangerV2 Initialized (GPU_NUM(cuda):0, mps_enabled:False, onnx_device:CPU-DML)
[Voice Changer][RVC]: update_settings gpu:0
[Voice Changer][RVCr2] Initializing...
2024-01-05 00:12:06.3732371 [W:onnxruntime:, session_state.cc:1169 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-01-05 00:12:06.3787072 [W:onnxruntime:, session_state.cc:1171 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
[Voice Changer] generate new embedder. (no embedder)
[Voice Changer] Loading index...
[Voice Changer] Index file is not found
GENERATE INFERENCER<voice_changer.RVC.inferencer.OnnxRVCInferencer.OnnxRVCInferencer object at 0x000001CD4AEEBDF0>
GENERATE EMBEDDER<voice_changer.RVC.embedder.OnnxContentvec.OnnxContentvec object at 0x000001CD4AEEB550>
GENERATE PITCH EXTRACTOR<voice_changer.RVC.pitchExtractor.RMVPEOnnxPitchExtractor.RMVPEOnnxPitchExtractor object at 0x000001CD4AEEBE50>
[Voice Changer] [RVC] Initializing... done
[Voice Changer][RVC]: update_settings serverReadChunkSize:192
[Voice Changer][RVC]: update_settings extraConvertSize:65536
[Voice Changer][RVC]: update_settings f0Detector:rmvpe_onnx
[Voice Changer][RVC]: update_settings silentThreshold:0.00046
[Voice Changer][RVC]: update_settings enableServerAudio:0
[Voice Changer][RVC]: update_settings modelSlotIndex:1704378035022
[Voice Changer] VoiceChangerManager initializing... done.
[Voice Changer] MMVC_Rest initializing...
[Voice Changer] MMVC_Rest initializing... done.
[Voice Changer] MMVC_SocketIOApp initializing...
[Voice Changer] MMVC_SocketIOApp initializing... done.
[2024-01-05 00:12:08] connet sid : E3TjDhEaf80qqkLhAAAC
[2024-01-05 00:12:08] connet sid : 9tbeTxEBfV--wW3CAAAD
[VCClient] wait web server... done 200
[2024-01-05 00:12:55] connet sid : 80SxrH0QRUcbearLAAAF
Generated Strengths: for prev:(4096,), for cur:(4096,)
[Voice Changer] warming up... generating sola buffer.
ArchPigeon commented 6 months ago

I am also having this issue, also with an AMD GPU

KebabLord commented 1 month ago

bump