Inference time

[puckikk1202]

Hi, I'm grateful for your excellent work! I've implemented the code as per the instructions, and it runs without errors. However, the inference time is slow, approximately 176 seconds per iteration. I tested it on an 80G A100 GPU, and it seems to be using around 71G of GPU memory. Is this normal?

[ShenhaoZhu]

The inference time and the GPU memory usage have significantly exceeded expectations. You could try terminating unrelated processes and then give it another try.

[G-force78]

Thats odd, with a google A100 for motion-06 its peaking at 12.2GB

100% 20/20 [01:21<00:00, 4.05s/it] 100% 116/116 [00:04<00:00, 27.11it/s]

[chengzeyi]

Thats odd, with a google A100 for motion-06 its peaking at 12.2GB

100% 20/20 [01:21<00:00, 4.05s/it] 100% 116/116 [00:04<00:00, 27.11it/s]

How did you get that? With an RTX 4090 I get much more VRAM usage than that number. Ok, that may be related a BUG in WSL2. But how did you achieve such a speed?

[G-force78]

You need a large system RAM too, I've just tried using the T4 on colab free tier but the system RAM maxed out at 12gb loading the motion module, maybe that can be sent to VRAM instead if you have high VRAM?

Here is my config file using motion-06

num_inference_steps: 20 guidance_scale: 6 enable_zero_snr: true weight_dtype: "fp16"

guidance_types:

• 'depth'
• 'normal'
• 'semantic_map'
• 'dwpose'

noise_scheduler_kwargs: num_train_timesteps: 1000 beta_start: 0.00085 beta_end: 0.012 beta_schedule: "linear" steps_offset: 1 clip_sample: false

unet_additional_kwargs: use_inflated_groupnorm: true unet_use_cross_frame_attention: false unet_use_temporal_attention: false use_motion_module: true motion_module_resolutions:

• 1
• 2
• 4
• 8 motion_module_mid_block: true motion_module_decoder_only: false motion_module_type: Vanilla motion_module_kwargs: num_attention_heads: 8 num_transformer_block: 1 attention_block_types:
  • Temporal_Self
  • Temporal_Self temporal_position_encoding: true temporal_position_encoding_max_len: 32 temporal_attention_dim_div: 1

guidance_encoder_kwargs: guidance_embedding_channels: 320 guidance_input_channels: 3 block_out_channels: [16, 32, 96, 256]

enable_xformers_memory_efficient_attention: true