Windows problems

[SoftologyPro]

Running app.py on Windows gives this error

[14:56:53] Load appearance_feature_extractor from D:\Tests\JoyVASA\JoyVASA\pretrained_weights\liveportrait\base_models\appearance_feature_extractor.pth done.                                                                                  live_portrait_wmg_wrapper.py:55
           Load motion_extractor from D:\Tests\JoyVASA\JoyVASA\pretrained_weights\liveportrait\base_models\motion_extractor.pth done.                                                                                                          live_portrait_wmg_wrapper.py:58
[14:56:54] Load warping_module from D:\Tests\JoyVASA\JoyVASA\pretrained_weights\liveportrait\base_models\warping_module.pth done.                                                                                                              live_portrait_wmg_wrapper.py:61
[14:56:55] Load spade_generator from D:\Tests\JoyVASA\JoyVASA\pretrained_weights\liveportrait\base_models\spade_generator.pth done.                                                                                                            live_portrait_wmg_wrapper.py:64
           Load stitching_retargeting_module from D:\Tests\JoyVASA\JoyVASA\pretrained_weights\liveportrait\retargeting_models\stitching_retargeting_module.pth done.                                                                           live_portrait_wmg_wrapper.py:68
Traceback (most recent call last):
  File "D:\Tests\JoyVASA\JoyVASA\app.py", line 48, in <module>
    gradio_pipeline_human = GradioPipeline(
  File "D:\Tests\JoyVASA\JoyVASA\src\gradio_pipeline.py", line 27, in __init__
    super().__init__(inference_cfg, crop_cfg)
  File "D:\Tests\JoyVASA\JoyVASA\src\live_portrait_wmg_pipeline.py", line 34, in __init__
    self.live_portrait_wrapper: LivePortraitWrapper = LivePortraitWrapper(inference_cfg=inference_cfg)
  File "D:\Tests\JoyVASA\JoyVASA\src\live_portrait_wmg_wrapper.py", line 81, in __init__
    self.motion_generator, self.motion_generator_args = load_model(inference_cfg.checkpoint_MotionGenerator, model_config, self.device, 'motion_generator')
  File "D:\Tests\JoyVASA\JoyVASA\src\utils\helper.py", line 161, in load_model
    model_data = torch.load(ckpt_path, map_location=device)
  File "D:\Tests\JoyVASA\JoyVASA\venv\lib\site-packages\torch\serialization.py", line 1026, in load
    return _load(opened_zipfile,
  File "D:\Tests\JoyVASA\JoyVASA\venv\lib\site-packages\torch\serialization.py", line 1438, in _load
    result = unpickler.load()
  File "D:\Python\lib\pathlib.py", line 962, in __new__
    raise NotImplementedError("cannot instantiate %r on your system"
NotImplementedError: cannot instantiate 'PosixPath' on your system

Which can be fixed by editing the start of app.py to redirect posixpath to windowsparh

from src.config.argument_config import ArgumentConfig
from src.config.inference_config import InferenceConfig

#add these next 3 lines
import pathlib
temp = pathlib.PosixPath
pathlib.PosixPath = pathlib.WindowsPath

def partial_fields(target_class, kwargs):
    return target_class(**{k: v for k, v in kwargs.items() if hasattr(target_class, k)})

But now when app.py is run it gives this error

[14:59:00] Load appearance_feature_extractor from D:\Tests\JoyVASA\JoyVASA\pretrained_weights\liveportrait\base_models\appearance_feature_extractor.pth done.                                                                                  live_portrait_wmg_wrapper.py:55
           Load motion_extractor from D:\Tests\JoyVASA\JoyVASA\pretrained_weights\liveportrait\base_models\motion_extractor.pth done.                                                                                                          live_portrait_wmg_wrapper.py:58
           Load warping_module from D:\Tests\JoyVASA\JoyVASA\pretrained_weights\liveportrait\base_models\warping_module.pth done.                                                                                                              live_portrait_wmg_wrapper.py:61
[14:59:01] Load spade_generator from D:\Tests\JoyVASA\JoyVASA\pretrained_weights\liveportrait\base_models\spade_generator.pth done.                                                                                                            live_portrait_wmg_wrapper.py:64
           Load stitching_retargeting_module from D:\Tests\JoyVASA\JoyVASA\pretrained_weights\liveportrait\retargeting_models\stitching_retargeting_module.pth done.                                                                           live_portrait_wmg_wrapper.py:68
using hubert chinese ori
Traceback (most recent call last):
  File "D:\Tests\JoyVASA\JoyVASA\venv\lib\site-packages\transformers\utils\hub.py", line 398, in cached_file
    resolved_file = hf_hub_download(
  File "D:\Tests\JoyVASA\JoyVASA\venv\lib\site-packages\huggingface_hub\utils\_validators.py", line 106, in _inner_fn
    validate_repo_id(arg_value)
  File "D:\Tests\JoyVASA\JoyVASA\venv\lib\site-packages\huggingface_hub\utils\_validators.py", line 160, in validate_repo_id
    raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'D:\Tests\JoyVASA\JoyVASA\pretrained_weights\TencentGameMate:chinese-hubert-base'.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "D:\Tests\JoyVASA\JoyVASA\app.py", line 48, in <module>
    gradio_pipeline_human = GradioPipeline(
  File "D:\Tests\JoyVASA\JoyVASA\src\gradio_pipeline.py", line 27, in __init__
    super().__init__(inference_cfg, crop_cfg)
  File "D:\Tests\JoyVASA\JoyVASA\src\live_portrait_wmg_pipeline.py", line 34, in __init__
    self.live_portrait_wrapper: LivePortraitWrapper = LivePortraitWrapper(inference_cfg=inference_cfg)
  File "D:\Tests\JoyVASA\JoyVASA\src\live_portrait_wmg_wrapper.py", line 81, in __init__
    self.motion_generator, self.motion_generator_args = load_model(inference_cfg.checkpoint_MotionGenerator, model_config, self.device, 'motion_generator')
  File "D:\Tests\JoyVASA\JoyVASA\src\utils\helper.py", line 163, in load_model
    model = DitTalkingHead(motion_feat_dim=model_args.motion_feat_dim,
  File "D:\Tests\JoyVASA\JoyVASA\src\modules\dit_talking_head.py", line 130, in __init__
    self.audio_encoder = HubertModel.from_pretrained(make_abs_path('../../pretrained_weights/TencentGameMate:chinese-hubert-base'))
  File "D:\Tests\JoyVASA\JoyVASA\venv\lib\site-packages\transformers\modeling_utils.py", line 2899, in from_pretrained
    resolved_config_file = cached_file(
  File "D:\Tests\JoyVASA\JoyVASA\venv\lib\site-packages\transformers\utils\hub.py", line 462, in cached_file
    raise EnvironmentError(
OSError: Incorrect path_or_model_id: 'D:\Tests\JoyVASA\JoyVASA\pretrained_weights\TencentGameMate:chinese-hubert-base'. Please provide either the path to a local folder or the repo_id of a model on the Hub.

How to fix for Windows?

[Kallamamran]

This is after adding the three lines for me:

(venv) Q:\JoyVASA>python inference.py -r face1.jpg -a face1.wav --animation_mode human --cfg_scale 2.0
[09:02:44] Load appearance_feature_extractor from                      live_portrait_wmg_wrapper.py:55
           Q:\JoyVASA\pretrained_weights\liveportrait\base_models\appe
           arance_feature_extractor.pth done.
           Load motion_extractor from                                  live_portrait_wmg_wrapper.py:58
           Q:\JoyVASA\pretrained_weights\liveportrait\base_models\moti
           on_extractor.pth done.
[09:02:45] Load warping_module from                                    live_portrait_wmg_wrapper.py:61
           Q:\JoyVASA\pretrained_weights\liveportrait\base_models\warp
           ing_module.pth done.
           Load spade_generator from                                   live_portrait_wmg_wrapper.py:64
           Q:\JoyVASA\pretrained_weights\liveportrait\base_models\spad
           e_generator.pth done.
           Load stitching_retargeting_module from                      live_portrait_wmg_wrapper.py:68
           Q:\JoyVASA\pretrained_weights\liveportrait\retargeting_mode
           ls\stitching_retargeting_module.pth done.
Traceback (most recent call last):
  File "Q:\JoyVASA\inference.py", line 67, in <module>
    main()
  File "Q:\JoyVASA\inference.py", line 55, in main
    pipeline = LivePortraitPipeline(
  File "Q:\JoyVASA\src\live_portrait_wmg_pipeline.py", line 34, in __init__
    self.live_portrait_wrapper: LivePortraitWrapper = LivePortraitWrapper(inference_cfg=inference_cfg)
  File "Q:\JoyVASA\src\live_portrait_wmg_wrapper.py", line 81, in __init__
    self.motion_generator, self.motion_generator_args = load_model(inference_cfg.checkpoint_MotionGenerator, model_config, self.device, 'motion_generator')
  File "Q:\JoyVASA\src\utils\helper.py", line 161, in load_model
    model_data = torch.load(ckpt_path, map_location=device)
  File "Q:\JoyVASA\venv\lib\site-packages\torch\serialization.py", line 1026, in load
    return _load(opened_zipfile,
  File "Q:\JoyVASA\venv\lib\site-packages\torch\serialization.py", line 1438, in _load
    result = unpickler.load()
  File "C:\Program Files\Python310\lib\pathlib.py", line 962, in __new__
    raise NotImplementedError("cannot instantiate %r on your system"
NotImplementedError: cannot instantiate 'PosixPath' on your system

[nitinmukesh]

I have raised PR for 'PosixPath' issue. You can manually replace the file for now.

[SoftologyPro]

I have raised PR for 'PosixPath' issue. You can manually replace the file for now.

OK, thanks. Can you raise a PR for the second error I show? That is the real problem that causes the app.py to not launch.

[DBDXSS]

I have raised PR for 'PosixPath' issue. You can manually replace the file for now.

The PR has been completed.

[nitinmukesh]

@DBDXSS I have raised another PR to fix the 2nd issue reported https://github.com/jdh-algo/JoyVASA/pull/8

No further updates required to work on Windows after this.

@SoftologyPro Please manually merge the file from PR and test. The 2nd issue should be fixed.

[DBDXSS]

@DBDXSS I have raised another PR to fix the 2nd issue reported #8

No further updates required to work on Windows after this.

@SoftologyPro Please manually merge the file from PR and test. The 2nd issue should be fixed.

Thanks! We have done that.

[nitinmukesh]

@SoftologyPro

Please verify and close if working.

@DBDXSS

Thank you

[nitinmukesh]

@DBDXSS

Sorry forgot to include Readme update. When you get time to merge.