(pytorch) root@notebook-1fd34197-03c6-4c8f-93b7-906197f1134f-0:~/Open-Sora# torchrun --standalone --nproc_per_node 1 scripts/inference.py configs/opensora/inference/16x256x256.py --ckpt-path OpenSora-v1-HQ-16x256x256.pth --prompt-path ./assets/texts/t2v_samples.txt /opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. _torch_pytree._register_pytree_node( /opt/conda/envs/pytorch/lib/python3.9/site-packages/colossalai/pipeline/schedule/_utils.py:19: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. _register_pytree_node(OrderedDict, _odict_flatten, _odict_unflatten) /root/.local/lib/python3.9/site-packages/torch/utils/_pytree.py:300: UserWarning: <class 'collections.OrderedDict'> is already registered as pytree node. Overwriting the previous registration. warnings.warn( /opt/conda/envs/pytorch/lib/python3.9/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/opt/conda/envs/pytorch/lib/python3.9/site-packages/torchvision/image.so: undefined symbol: _ZN3c1017RegisterOperatorsD1Ev'If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source? warn( /opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/utils/generic.py:309: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. _torch_pytree._register_pytree_node( /opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/utils/generic.py:309: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. _torch_pytree._register_pytree_node( Config (path: configs/opensora/inference/16x256x256.py): {'num_frames': 16, 'fps': 8, 'image_size': (256, 256), 'model': {'type': 'STDiT-XL/2', 'space_scale': 0.5, 'time_scale': 1.0, 'enable_flashattn': True, 'enable_layernorm_kernel': True, 'from_pretrained': 'OpenSora-v1-HQ-16x256x256.pth'}, 'vae': {'type': 'VideoAutoencoderKL', 'from_pretrained': 'stabilityai/sd-vae-ft-ema', 'micro_batch_size': 4}, 'text_encoder': {'type': 't5', 'from_pretrained': 'DeepFloyd/t5-v1_1-xxl', 'model_max_length': 120}, 'scheduler': {'type': 'iddpm', 'num_sampling_steps': 100, 'cfg_scale': 7.0, 'cfg_channel': 3}, 'dtype': 'bf16', 'prompt_path': './assets/texts/t2v_samples.txt', 'prompt': ["A soaring drone footage captures the majestic beauty of a coastal cliff, its red and yellow stratified rock faces rich in color and against the vibrant turquoise of the sea. Seabirds can be seen taking flight around the cliff's precipices. As the drone slowly moves from different angles, the changing sunlight casts shifting shadows that highlight the rugged textures of the cliff and the surrounding calm sea. The water gently laps at the rock base and the greenery that clings to the top of the cliff, and the scene gives a sense of peaceful isolation at the fringes of the ocean. The video captures the essence of pristine natural beauty untouched by human structures.", "A majestic beauty of a waterfall cascading down a cliff into a serene lake. The waterfall, with its powerful flow, is the central focus of the video. The surrounding landscape is lush and green, with trees and foliage adding to the natural beauty of the scene. The camera angle provides a bird's eye view of the waterfall, allowing viewers to appreciate the full height and grandeur of the waterfall. The video is a stunning representation of nature's power and beauty.", 'A vibrant scene of a snowy mountain landscape. The sky is filled with a multitude of colorful hot air balloons, each floating at different heights, creating a dynamic and lively atmosphere. The balloons are scattered across the sky, some closer to the viewer, others further away, adding depth to the scene. Below, the mountainous terrain is blanketed in a thick layer of snow, with a few patches of bare earth visible here and there. The snow-covered mountains provide a stark contrast to the colorful balloons, enhancing the visual appeal of the scene. In the foreground, a few cars can be seen driving along a winding road that cuts through the mountains. The cars are small compared to the vastness of the landscape, emphasizing the grandeur of the surroundings. The overall style of the video is a mix of adventure and tranquility, with the hot air balloons adding a touch of whimsy to the otherwise serene mountain landscape. The video is likely shot during the day, as the lighting is bright and even, casting soft shadows on the snow-covered mountains.', "The vibrant beauty of a sunflower field. The sunflowers, with their bright yellow petals and dark brown centers, are in full bloom, creating a stunning contrast against the green leaves and stems. The sunflowers are arranged in neat rows, creating a sense of order and symmetry. The sun is shining brightly, casting a warm glow on the flowers and highlighting their intricate details. The video is shot from a low angle, looking up at the sunflowers, which adds a sense of grandeur and awe to the scene. The sunflowers are the main focus of the video, with no other objects or people present. The video is a celebration of nature's beauty and the simple joy of a sunny day in the countryside.", "A serene underwater scene featuring a sea turtle swimming through a coral reef. The turtle, with its greenish-brown shell, is the main focus of the video, swimming gracefully towards the right side of the frame. The coral reef, teeming with life, is visible in the background, providing a vibrant and colorful backdrop to the turtle's journey. Several small fish, darting around the turtle, add a sense of movement and dynamism to the scene. The video is shot from a slightly elevated angle, providing a comprehensive view of the turtle's surroundings. The overall style of the video is calm and peaceful, capturing the beauty and tranquility of the underwater world.", 'A vibrant underwater scene. A group of blue fish, with yellow fins, are swimming around a coral reef. The coral reef is a mix of brown and green, providing a natural habitat for the fish. The water is a deep blue, indicating a depth of around 30 feet. The fish are swimming in a circular pattern around the coral reef, indicating a sense of motion and activity. The overall scene is a beautiful representation of marine life.', "A bustling city street at night, filled with the glow of car headlights and the ambient light of streetlights. The scene is a blur of motion, with cars speeding by and pedestrians navigating the crosswalks. The cityscape is a mix of towering buildings and illuminated signs, creating a vibrant and dynamic atmosphere. The perspective of the video is from a high angle, providing a bird's eye view of the street and its surroundings. The overall style of the video is dynamic and energetic, capturing the essence of urban life at night.", 'A snowy forest landscape with a dirt road running through it. The road is flanked by trees covered in snow, and the ground is also covered in snow. The sun is shining, creating a bright and serene atmosphere. The road appears to be empty, and there are no people or animals visible in the video. The style of the video is a natural landscape shot, with a focus on the beauty of the snowy forest and the peacefulness of the road.', 'The dynamic movement of tall, wispy grasses swaying in the wind. The sky above is filled with clouds, creating a dramatic backdrop. The sunlight pierces through the clouds, casting a warm glow on the scene. The grasses are a mix of green and brown, indicating a change in seasons. The overall style of the video is naturalistic, capturing the beauty of the landscape in a realistic manner. The focus is on the grasses and their movement, with the sky serving as a secondary element. The video does not contain any human or animal elements.', 'A serene night scene in a forested area. The first frame shows a tranquil lake reflecting the star-filled sky above. The second frame reveals a beautiful sunset, casting a warm glow over the landscape. The third frame showcases the night sky, filled with stars and a vibrant Milky Way galaxy. The video is a time-lapse, capturing the transition from day to night, with the lake and forest serving as a constant backdrop. The style of the video is naturalistic, emphasizing the beauty of the night sky and the peacefulness of the forest.'], 'batch_size': 1, 'seed': 42, 'save_dir': './samples/samples/', 'config': 'configs/opensora/inference/16x256x256.py', 'prompt_as_path': False, 'reference_path': None, 'loop': 1, 'frame_interval': 1, 'sample_name': None, 'num_sample': 1, 'multi_resolution': False} /opt/conda/envs/pytorch/lib/python3.9/site-packages/colossalai/initialize.py:48: UserWarning: config is deprecated and will be removed soon. warnings.warn("config is deprecated and will be removed soon.") [06/21/24 03:35:58] INFO colossalai - colossalai - INFO: /opt/conda/envs/pytorch/lib/python3.9/site-packages/colossalai/initialize.py:67 launch
INFO colossalai - colossalai - INFO: Distributed environment is initialized, world size: 1
Downloading shards: 0%| | 0/2 [00:00<?, ?it/s]/opt/conda/envs/pytorch/lib/python3.9/site-packages/huggingface_hub/file_download.py:1003: UserWarning: Not enough free disk space to download the file. The expected file size is: 9597.03 MB. The target location /root/.cache/huggingface/hub only has 0.00 MB free disk space. warnings.warn( /opt/conda/envs/pytorch/lib/python3.9/site-packages/huggingface_hub/file_download.py:1003: UserWarning: Not enough free disk space to download the file. The expected file size is: 9597.03 MB. The target location /root/.cache/huggingface/hub/models--DeepFloyd--t5-v1_1-xxl/blobs only has 0.00 MB free disk space. warnings.warn( Downloading shards: 50%|████████████████████████████████████████████████████████████████████████████ | 1/2 00:02<00:02, 2.56s/it: Traceback (most recent call last): | 10.5M/9.60G 00:01<20:21, 7.85MB/s: File "/root/Open-Sora/scripts/inference.py", line 174, in

rank0: File "/root/Open-Sora/scripts/inference.py", line 57, in main rank0: text_encoder = build_module(cfg.text_encoder, MODELS, device=device) # T5 must be fp32 rank0: File "/root/.local/lib/python3.9/site-packages/opensora/registry.py", line 22, in build_module rank0: return builder.build(cfg) rank0: File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/mmengine/registry/registry.py", line 570, in build rank0: return self.build_func(cfg, args, kwargs, registry=self) rank0: File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg rank0: obj = obj_cls(args) # type: ignore rank0: File "/root/.local/lib/python3.9/site-packages/opensora/models/text_encoder/t5.py", line 151, in init rank0: self.t5 = T5Embedder( rank0: File "/root/.local/lib/python3.9/site-packages/opensora/models/text_encoder/t5.py", line 108, in init rank0: self.model = T5EncoderModel.from_pretrained( rank0: File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/modeling_utils.py", line 3351, in from_pretrained rank0: resolved_archive_file, sharded_metadata = get_checkpoint_shard_files( rank0: File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/utils/hub.py", line 1017, in get_checkpoint_shard_files rank0: cached_filename = cached_file( rank0: File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/transformers/utils/hub.py", line 389, in cached_file rank0: resolved_file = hf_hub_download( rank0: File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 119, in _inner_fn rank0: return fn(args, **kwargs) rank0: File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/huggingface_hub/file_download.py", line 1492, in hf_hub_download

rank0: File "/opt/conda/envs/pytorch/lib/python3.9/site-packages/huggingface_hub/file_download.py", line 538, in http_get

api.py", line 263, in launch_agent raise ChildFailedError( torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

scripts/inference.py FAILED

Failures:

------------------------------------------------------------ Root Cause (first observed failure): [0]: time : 2024-06-21_03:36:09 host : notebook-1fd34197-03c6-4c8f-93b7-906197f1134f-0.notebook-1fd34197-03c6-4c8f-93b7-906197f1134f.colossal-ai.svc.cluster.local rank : 0 (local_rank: 0) exitcode : 1 (pid: 1232) error_file: traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html 是什么原因呢？

hpcaitech / Open-Sora

在潞晨云尝试跑 1.1版本，按照 READEME1.1 运行报错 #516

scripts/inference.py FAILED