Request for .npz Output Only and Blender Error

hyunaseo commented 3 weeks ago

Thanks for the interesting work!

I'm currently trying to use the results of Amuse in the Unity engine. I'd like to obtain only the results from infer_gesture as an .npz file. Currently, running infer_gesture triggers the full pipeline, including launching Blender. Is there a way to obtain only the gesture generation results without having to execute Blender?
Also, when I run infer_gesture, Blender is launched automatically, but I keep encountering the following error. Do you have any idea what might be causing this?

[LATDIFF - INFER AST] Frame based feats flag: True
[LATDIFF] (2/3) <===== Chosen AST model:  /app/EmotionAR/amuse/saved-models/wav_dtw_mfcc_20231022-044436_actors/model_1_tL1.18493436_tEA97.78573690_tPA97.82135507_vL39.93227725_vEA83.59375000_vPA0.00000000.pkl  , loading state dict... =====>
Blender 3.4.1 (hash 55485cb379f7 built 2022-12-20 00:46:45)
Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = (not set)
  program name = '/home/kchhatre/Work/blender-3.4.1-linux-x64/blender'
  isolated = 0
  environment = 1
  user site = 0
  import site = 1
  sys._base_executable = '/opt/conda/envs/amuse/bin/python'
  sys.base_prefix = '/src/build_linux/deps/Release/python'
  sys.base_exec_prefix = '/src/build_linux/deps/Release/python'
  sys.platlibdir = 'lib'
  sys.executable = '/opt/conda/envs/amuse/bin/python'
  sys.prefix = '/src/build_linux/deps/Release/python'
  sys.exec_prefix = '/src/build_linux/deps/Release/python'
  sys.path = [
    '/src/build_linux/deps/Release/python/lib/python310.zip',
    '/src/build_linux/deps/Release/python/lib/python3.10',
    '/src/build_linux/deps/Release/python/lib/lib-dynload',
  ]
Internal error initializing Python!
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007f4dd2070240 (most recent call first):
  <no Python frame>
[BLENDER] setup complete!
 <===== INIT: AUDIO LIST LPDM EVALUATION, REP 1/1 =====>
[LDM EVAL] Audio list inference done, total time elapsed: 0.0004 s
AMUSE: (infer_gesture) completed in: 0.0033825136555565728 hrs
(amuse) root@5146db9f50f9:/app/EmotionAR/amuse/scripts# python main.py --fn bvh2smplx_
[MOVE LMDBs] old cache:  2023-10-28_30F_fing_smplx_MOSH_full_v1_feat_based_300  on local machine
[MOVE LMDBs] found relevant caches only, skipping...
Experiment init: AMUSE, process: 3318, running on: 5146db9f50f9, distributed: none, device: cuda, time: Tue Nov  5 14:52:57 2024
[RAW PATHS/ LATDIFF] Cache already exists. Skipping cache process.
[RAW PATHS] All English data pickles already exist, loading...
[RAW PATHS] All English pickle data loaded. Total datapoints:  1941
[Beat2SMPLFBX] Converting BVH to SMPLX NPZ...
processing 1/1: /app/EmotionAR/amuse/data/beat-rawdata-eng/beat_rawdata_english/16/16_jorge_0_3_3.bvh
[Beat2SMPLNPZ] Conversion finished in 2.578894297281901e-06 minutes
AMUSE: Smplx npz conversion done! Import NPZ in Blender with SMPLX addon and specify correct fps.

Thank you!

kiranchhatre commented 3 weeks ago

To extract only the .npz file, you can comment out the line self.visualizer.load_in_blender(self.EXEC_ON_CLUSTER) in the trainer's __init__ method here. Since the composition happens in the Visualizer class, you can inherit from the original Visualizer class in visualizer.py and modify the functions that compose the .npz files (animate_ldm_sample_v1, animate_ldm_sample_v2) without the Blender render subprocess calls.
This issue will disappear if you do step 1.

Let me know if this works.

hyunaseo commented 3 weeks ago

Hi!! I resolved the blender issue "somewhat" haha 😂 Now, the blender executes, but there's a plugin issue -- after face baker, it is terminated. Do you know what is this?

MHX2 successfully (re)loaded Loading BVH Retargeter BVH Retargeter loaded 0 meshes freed [BLENDER] Already enabled: Stop-motion-OBJ [BLENDER] Already enabled: facebaker [BLENDER] Already enabled: import_runtime_mhx2 [BLENDER] Already enabled: retarget_bvh [BLENDER] Already enabled: auto_rig_pro-master [BLENDER] smplx_blender_addon is not enabled. Enabling... Writing userprefs: '/home/hyunaseo/.config/blender/3.4/config/userpref.blend' ok Info: Preferences saved [FACEBAKER] Disable plugin

Blender quit [BLENDER] setup complete! <===== INIT: AUDIO LIST LPDM EVALUATION, REP 1/1 =====> [LDM EVAL] Audio list inference done, total time elapsed: 0.0004 s AMUSE: (infer_gesture) completed in: 0.0036576798227098252 hrs

kiranchhatre / amuse

Request for .npz Output Only and Blender Error #12