adapted pretrained model to training

hpcaitech / Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

https://hpcaitech.github.io/Open-Sora/

Apache License 2.0

21.48k stars 2.06k forks source link

adapted pretrained model to training #371

Closed FrankLeeeee closed 3 months ago

budzianowski commented 3 months ago

Does this allows to use pretrained model models--hpcai-tech--OpenSora-STDiT-v2-stage3/snapshots/3074d7cc95aca24ae12b685e07fab7d3e2570667/model.safetensors for finetuning? This does not work: torchrun --nnodes=1 --nproc_per_node=8 scripts /train.py ... -load /home/pawelx/.cache/huggingface/hub/models--hpcai-tech--OpenSora-STDiT-v2-stage3/snapshots/3074d7cc95aca24ae12b685e07fab7d3e2570667/model.safetensors I assume lack of ema will be a problem as well?

FrankLeeeee commented 3 months ago

We have updated the config to support huggingface's from_pretrained, you can specify from pretrained like this https://github.com/hpcaitech/Open-Sora/blob/main/configs/opensora-v1-1/inference/sample.py#L10.