Not working

[Tanmaypatil123]

I am trying to infer it with four h100, and after running the gradio demo, it was giving this error. can you please check and simplify the inferencing.

[Filaind]

(.venv) root@pi-0:~/models/src/mochi_preview# python infer.py --prompt "A hand with delicate fingers picks up a bright yellow lemon from a wooden bowl filled with lemons and sprigs of mint against a peach-colored background. The hand gently tosses the lemon up and catches it, showcasing its smooth texture. A beige string bag sits beside the bowl, adding a rustic touch to the scene. Additional lemons, one halved, are scattered around the base of the bowl. The even lighting enhances the vibrant colors and creates a fresh, inviting atmosphere." --seed 1710977262 --model_dir "/root/test" 2024-10-22 20:26:56,033 WARNING utils.py:580 -- Detecting docker specified CPUs. In previous versions of Ray, CPU detection in containers was incorrect. Please ensure that Ray has enough CPUs allocated. As a temporary workaround to revert to the prior behavior, set RAY_USE_MULTIPROCESSING_CPU_COUNT=1 as an env var before starting Ray. Set the env var: RAY_DISABLE_DOCKER_CPU_WARNING=1 to mute this warning. 2024-10-22 20:26:56,160 INFO worker.py:1786 -- Started a local Ray instance. (T2VSynthMochiModel pid=12841) Timing init_process_group

Stuck here

[nousr]

+1 -- the cli also errored out with a similar error

8xh100s - with cuda12.5 on a fresh docker container

[victorchall]

I got it mostly working on a single 48GB instance. Working on the fork...

[nousr]

hey! @victorchall long time no see

nice work! looking forward to seeing the fork 😄

[ved-genmo]

@Tanmaypatil123 Sorry about that! The error should be fixed now. There was a bug where the code assumed the user has 8 GPUs.

@Filaind The init_process_group can take 20-3 seconds, does it still stall after that?

[victorchall]

Here ya go: https://github.com/victorchall/genmoai-smol num_workers is hard coded to 8 in the original, I set it to 1 and fixed a few minor problems that arise by not using torch.dist.

This likely breaks >1 gpu/num_workers. I guess some toggles could be added to the ui.

I'm shifting t5, vae, etc back and forth from cpu to gpu when not in use and using bfloat16 across the board for weights and AMP. 73 frames and otherwise default settings takes 26GB VRAM (848x480), so 67 or 61 frames might even fit into 24GB. 73 frames at 100 inference steps takes about 15 minutes on an RTX 6000 Ada. I also forced the frame count to the allowed t-1%6==0

[zixian-wu]

(.venv) root@pi-0:~/models/src/mochi_preview# python infer.py --prompt "A hand with delicate fingers picks up a bright yellow lemon from a wooden bowl filled with lemons and sprigs of mint against a peach-colored background. The hand gently tosses the lemon up and catches it, showcasing its smooth texture. A beige string bag sits beside the bowl, adding a rustic touch to the scene. Additional lemons, one halved, are scattered around the base of the bowl. The even lighting enhances the vibrant colors and creates a fresh, inviting atmosphere." --seed 1710977262 --model_dir "/root/test" 2024-10-22 20:26:56,033 WARNING utils.py:580 -- Detecting docker specified CPUs. In previous versions of Ray, CPU detection in containers was incorrect. Please ensure that Ray has enough CPUs allocated. As a temporary workaround to revert to the prior behavior, set RAY_USE_MULTIPROCESSING_CPU_COUNT=1 as an env var before starting Ray. Set the env var: RAY_DISABLE_DOCKER_CPU_WARNING=1 to mute this warning. 2024-10-22 20:26:56,160 INFO worker.py:1786 -- Started a local Ray instance. (T2VSynthMochiModel pid=12841) Timing init_process_group

Stuck here

have the same issue