search

PKU-YuanGroup / Open-Sora-Plan

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.
MIT License
11.46k stars 1.02k forks source link

Clarification Needed: Which model for Inference from fine tune Checkpoint Structure #502

Open yachty66 opened 3 hours ago

yachty66 commented 3 hours ago

I completed a small fine-tuning with dummy data. The checkpoints of the model are saved as follows:

checkpoint
│
├── model_ema
│   ├── diffusion_pytorch_..._fetensors.index.json
│   ├── diffusion_pytorch_..._f-00002.safetensors
│   ├── diffusion_pytorch_..._f-00002.safetensors
│   └── config.json
│
├── pytorch_model
│   ├── mp_rank_00_model_states.pt
│   ├── bf16_zero_pp_rank_..._00_optim_states.pt
│   └── bf16_zero_pp_rank_..._00_optim_states.pt
│
├── model
│   ├── config.json
│   ├── diffusion_pytorch_model.safetensors
│   ├── zero_to_fp32.py
│   ├── scheduler.bin
│   ├── random_states_1.pkl
│   ├── random_states_0.pkl
│   └── latest

When I want to run the checkpoint result from my fine tune on inference, what do I have to use as the main model and what as VAE?

LinB203 commented 2 hours ago

model_ema or model are both ok.