关于训练sd3，出现TypeError: JointTransformerBlock.forward() got an unexpected keyword argument 'offload_to_cpu'

[jiashenggu]

我启动lumina_next_t2i_mini/train_dreambooth_sd3.py，会报这个错 TypeError: JointTransformerBlock.forward() got an unexpected keyword argument 'offload_to_cpu' 我的启动脚本

torchrun --nproc-per-node=1 train_dreambooth_sd3.py \
    --master_port 18181 \
    --data_path ${train_data_root} \
    --results_dir results/${exp_name} \
    --micro_batch_size 4 \
    --global_batch_size ${batch_size} --lr ${lr} \
    --data_parallel sdp \
    --max_steps 1000 \
    --ckpt_every 1000 --log_every 10 \
    --use_t5 \
    --checkpointing \
    --precision ${precision} --grad_precision fp32 \
    --image_size ${image_size} \
    --model_path ${model_path} \
    --global_seed 3 \
    2>&1 | tee -a results/"$exp_name"/output_dev_"$datetime".log

顺便， https://github.com/Alpha-VLLM/Lumina-T2X/blob/2e7c7319b1b4b3dc7939f78bd0eeffa3c13822d2/lumina_t2i/train.py#L514 这是打错了吗？hsdp

[jiashenggu]

JointTransformerBlock.forward()加一个**kwargs可以解决，虽然不清楚有没有隐患，反正有人这么干了 https://github.com/huggingface/transformers/issues/30523