search

YuliangXiu / PuzzleAvatar

[SIGGRAPH Asia 2024] PuzzleAvatar: Assembling 3D Avatars from Personal Albums
https://puzzleavatar.is.tue.mpg.de/
Other
238 stars 10 forks source link

Help with completing the installation? #1

Open EightiesPower opened 1 month ago

EightiesPower commented 1 month ago

So, I've been trying to set this up for the last couple of hours and after following the instructions to best of my ability, here are the errors that come up when I run run.sh: 

Traceback (most recent call last):
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/multi_concepts/grounding_dino_sam.py", line 20, in <module>
    from groundingdino.util.inference import Model
ModuleNotFoundError: No module named 'groundingdino'
Traceback (most recent call last):
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/multi_concepts/islands_all.py", line 9, in <module>
    import rpack
ModuleNotFoundError: No module named 'rpack'

Traceback (most recent call last):
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/multi_concepts/train.py", line 62, in <module>
    import wandb
ModuleNotFoundError: No module named 'wandb'

Traceback (most recent call last):
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/multi_concepts/inference.py", line 26, in <module>
    from peft import PeftModel
ModuleNotFoundError: No module named 'peft'

Traceback (most recent call last):
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/cores/main_mc.py", line 12, in <module>
    from cores.lib.renderer import Renderer
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/cores/lib/renderer.py", line 6, in <module>
    import nvdiffrast.torch as dr
ModuleNotFoundError: No module named 'nvdiffrast'

Traceback (most recent call last):
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/utils/body_utils/postprocess_mc.py", line 25, in <module>
    smplx_path = glob(smplx_paths)[0]
IndexError: list index out of range

Traceback (most recent call last):
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/cores/main_mc.py", line 12, in <module>
    from cores.lib.renderer import Renderer
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/cores/lib/renderer.py", line 6, in <module>
    import nvdiffrast.torch as dr
ModuleNotFoundError: No module named 'nvdiffrast'

Now, I did install nvdiffrast. peft, and groundingdino. I've cloned each repository into the "thirdparties" folder and I've got groundingdino_swint_ogc and sam_vit_h_4b8939 in the PuzzleAvatar/weights folder. Can you tell what to install and where in order to move forward? Thank you.

YuliangXiu commented 1 month ago

Fixed, now you can git clone --recurse-submodules git@github.com:YuliangXiu/PuzzleAvatar.git to get all the submodules.

gt732 commented 1 month ago

@YuliangXiu will this work on windows?

YuliangXiu commented 1 month ago

@YuliangXiu will this work on windows?

I have not tested it on Windows, but it should work as long as you can setup the CUDA env.

gt732 commented 1 month ago

@YuliangXiu will this work on windows?

I have not tested it on Windows, but it should work as long as you can setup the CUDA env.

I will give it a shot. Ty for your hard work on this. Do you know if using this library I can achieve what viggle ai does locally?

https://viggle.ai/m-home

Looks like they create a 3D mesh using the reference image, then apply the extracted mocap data to the avatar. I may be wrong though I’m new to this stuff.

Thanks!!

EightiesPower commented 1 month ago

it clone --recurse-submodules git@github.com:YuliangXiu/PuzzleAvatar.git

Thanks for the response. I've done that and redid the Build modules section of the install guide. I'm still getting the following errors, however:

cat: OPENAI_API_KEY: No such file or directory
./scripts/env.sh: line 17: /home/yxiu/miniconda3/bin/activate: No such file or directory
Traceback (most recent call last):
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/multi_concepts/grounding_dino_sam.py", line 20, in <module>
    from groundingdino.util.inference import Model
ModuleNotFoundError: No module named 'groundingdino'
Traceback (most recent call last):
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/multi_concepts/islands_all.py", line 9, in <module>
    import rpack
ModuleNotFoundError: No module named 'rpack'
There was a problem when trying to write in your cache folder (/is/cluster/yxiu/.cache/hub). Please, ensure the directory exists and can be written to.
@torch.library.impl_abstract("xformers_flash::flash_bwd")
There was a problem when trying to write in your cache folder (/is/cluster/yxiu/.cache/hub). You should set the environment variable TRANSFORMERS_CACHE to a writable directory.
Traceback (most recent call last):
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/multi_concepts/train.py", line 62, in <module>
    import wandb
ModuleNotFoundError: No module named 'wandb'
There was a problem when trying to write in your cache folder (/is/cluster/yxiu/.cache/hub). Please, ensure the directory exists and can be written to.
 @torch.library.impl_abstract("xformers_flash::flash_bwd")
There was a problem when trying to write in your cache folder (/is/cluster/yxiu/.cache/hub). You should set the environment variable TRANSFORMERS_CACHE to a writable directory.
Traceback (most recent call last):
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/multi_concepts/inference.py", line 215, in <module>
    break_a_scene_inference = BreakASceneInference()
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/multi_concepts/inference.py", line 41, in __init__
    self._load_pipeline()
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/multi_concepts/inference.py", line 113, in _load_pipeline
    self.pipeline = DiffusionPipeline.from_pretrained(
  File "/home/eightiespower/miniconda3/envs/PuzzleAvatar/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/eightiespower/miniconda3/envs/PuzzleAvatar/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 719, in from_pretrained
    config_dict = cls.load_config(cached_folder)
  File "/home/eightiespower/miniconda3/envs/PuzzleAvatar/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/eightiespower/miniconda3/envs/PuzzleAvatar/lib/python3.10/site-packages/diffusers/configuration_utils.py", line 373, in load_config
    raise EnvironmentError(
OSError: Error no file named model_index.json found in directory results/human/yuliang.
  @torch.cuda.amp.autocast(enabled=False)
Traceback (most recent call last):
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/cores/main_mc.py", line 187, in <module>
    with open(json_path, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: './data/yuliang/gpt4v_simple.json'
Traceback (most recent call last):
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/utils/body_utils/postprocess_mc.py", line 25, in <module>
    smplx_path = glob(smplx_paths)[0]
IndexError: list index out of range
  @torch.cuda.amp.autocast(enabled=False)
Traceback (most recent call last):
  File "/media/eightiespower/01DB0091C59E37A0/AI_stuff/PuzzleAvatar/cores/main_mc.py", line 187, in <module>
    with open(json_path, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: './data/yuliang/gpt4v_simple.json'
EightiesPower commented 1 month ago

For reference I'm attaching a .txt listing the contents of my thirdparties folder. thirdparties folder contents.txt

EightiesPower commented 1 month ago

I also had the following errors when running bash scripts/download_body_data.sh

Username/Password Authentication Failed.
Archive:  data/body_data/smpl_related/models/SMPL_python_v.1.0.0.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of data/body_data/smpl_related/models/SMPL_python_v.1.0.0.zip or
        data/body_data/smpl_related/models/SMPL_python_v.1.0.0.zip.zip, and cannot find data/body_data/smpl_related/models/SMPL_python_v.1.0.0.zip.ZIP, period.
mv: cannot stat 'data/body_data/smpl_related/models/smpl/models/basicModel_f_lbs_10_207_0_v1.0.0.pkl': No such file or directory
mv: cannot stat 'data/body_data/smpl_related/models/smpl/models/basicmodel_m_lbs_10_207_0_v1.0.0.pkl': No such file or directory

Downloading SMPLify...
--2024-09-19 13:45:31--  https://download.is.tue.mpg.de/download.php?domain=smplify&sfile=mpips_smplify_public_v2.zip&resume=1
Resolving download.is.tue.mpg.de (download.is.tue.mpg.de)... 192.124.27.139
Connecting to download.is.tue.mpg.de (download.is.tue.mpg.de)|192.124.27.139|:443... connected.
HTTP request sent, awaiting response... 401 Unauthorized

Username/Password Authentication Failed.
Archive:  data/body_data/smpl_related/models/mpips_smplify_public_v2.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of data/body_data/smpl_related/models/mpips_smplify_public_v2.zip or
        data/body_data/smpl_related/models/mpips_smplify_public_v2.zip.zip, and cannot find data/body_data/smpl_related/models/mpips_smplify_public_v2.zip.ZIP, period.
mv: cannot stat 'data/body_data/smpl_related/models/smplify_public/code/models/basicModel_neutral_lbs_10_207_0_v1.0.0.pkl': No such file or directory

Downloading SMPL-X...
--2024-09-19 13:45:32--  https://download.is.tue.mpg.de/download.php?domain=smplx&sfile=models_smplx_v1_1.zip&resume=1
Resolving download.is.tue.mpg.de (download.is.tue.mpg.de)... 192.124.27.139
Connecting to download.is.tue.mpg.de (download.is.tue.mpg.de)|192.124.27.139|:443... connected.
HTTP request sent, awaiting response... 401 Unauthorized

Username/Password Authentication Failed.
Archive:  data/body_data/smpl_related/models/models_smplx_v1_1.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of data/body_data/smpl_related/models/models_smplx_v1_1.zip or
        data/body_data/smpl_related/models/models_smplx_v1_1.zip.zip, and cannot find data/body_data/smpl_related/models/models_smplx_v1_1.zip.ZIP, period.

--2024-09-19 14:02:22--  http://./
Resolving . (.)... failed: No address associated with hostname.
wget: unable to resolve host address ‘.’
gt732 commented 1 month ago

@EightiesPower I'm running through the installation as well.

This is how you get rpack its under a different name

https://rectangle-packer.readthedocs.io/en/latest/_modules/rpack.html

YuliangXiu commented 1 month ago

I also had the following errors when running bash scripts/download_body_data.sh

Username/Password Authentication Failed.
Archive:  data/body_data/smpl_related/models/SMPL_python_v.1.0.0.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of data/body_data/smpl_related/models/SMPL_python_v.1.0.0.zip or
        data/body_data/smpl_related/models/SMPL_python_v.1.0.0.zip.zip, and cannot find data/body_data/smpl_related/models/SMPL_python_v.1.0.0.zip.ZIP, period.
mv: cannot stat 'data/body_data/smpl_related/models/smpl/models/basicModel_f_lbs_10_207_0_v1.0.0.pkl': No such file or directory
mv: cannot stat 'data/body_data/smpl_related/models/smpl/models/basicmodel_m_lbs_10_207_0_v1.0.0.pkl': No such file or directory

Downloading SMPLify...
--2024-09-19 13:45:31--  https://download.is.tue.mpg.de/download.php?domain=smplify&sfile=mpips_smplify_public_v2.zip&resume=1
Resolving download.is.tue.mpg.de (download.is.tue.mpg.de)... 192.124.27.139
Connecting to download.is.tue.mpg.de (download.is.tue.mpg.de)|192.124.27.139|:443... connected.
HTTP request sent, awaiting response... 401 Unauthorized

Username/Password Authentication Failed.
Archive:  data/body_data/smpl_related/models/mpips_smplify_public_v2.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of data/body_data/smpl_related/models/mpips_smplify_public_v2.zip or
        data/body_data/smpl_related/models/mpips_smplify_public_v2.zip.zip, and cannot find data/body_data/smpl_related/models/mpips_smplify_public_v2.zip.ZIP, period.
mv: cannot stat 'data/body_data/smpl_related/models/smplify_public/code/models/basicModel_neutral_lbs_10_207_0_v1.0.0.pkl': No such file or directory

Downloading SMPL-X...
--2024-09-19 13:45:32--  https://download.is.tue.mpg.de/download.php?domain=smplx&sfile=models_smplx_v1_1.zip&resume=1
Resolving download.is.tue.mpg.de (download.is.tue.mpg.de)... 192.124.27.139
Connecting to download.is.tue.mpg.de (download.is.tue.mpg.de)|192.124.27.139|:443... connected.
HTTP request sent, awaiting response... 401 Unauthorized

Username/Password Authentication Failed.
Archive:  data/body_data/smpl_related/models/models_smplx_v1_1.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of data/body_data/smpl_related/models/models_smplx_v1_1.zip or
        data/body_data/smpl_related/models/models_smplx_v1_1.zip.zip, and cannot find data/body_data/smpl_related/models/models_smplx_v1_1.zip.ZIP, period.

--2024-09-19 14:02:22--  http://./
Resolving . (.)... failed: No address associated with hostname.
wget: unable to resolve host address ‘.’

You should firstly register at icon.is.tue.mpg.de, following register-at-icons-website, and input the username and password when you run the bash scripts.