[P1] TypeError: Object of type type is not JSON serializable

[srn-source]

I try to run train.py of alpaca examples with " TinyLlama/TinyLlama-1.1B-Chat-v1.0" but i got this error before training.

what should i do? please guide me. I am searching for it, i think it is about model.config cannot convert to json? i am not sure

[PinetreePantry]

Could you paste here the command or the script that you are running? I think the issue is related to Pyvene unable to serialize the config of the model that you are using

[srn-source]

@PinetreePantry i use the same in readme demo.

python train.py --model_name_or_path TinyLlama/TinyLlama-1.1B-Chat-v1.0 \ --data_path ./alpaca_data.json \ --output_dir ./test/ \ --layers "8;19" \ --rank 4 \ --position "f1+l1" \ --num_train_epochs 1 \ --per_device_train_batch_size 4 \ --per_device_eval_batch_size 4 \ --gradient_accumulation_steps 8 \ --evaluation_strategy "no" \ --save_strategy "no" \ --learning_rate 2e-5 \ --weight_decay 0. \ --warmup_ratio 0.03 \ --lr_scheduler_type "cosine" \ --logging_steps 1

[frankaging]

hey @srn-source, please try to add --report_to none and rerun. Thanks!

[srn-source]

@frankaging Its work, what happen? but thank you so much.

[PinetreePantry]

I think the reason might be because you accidentally enabled (probably by default) tensorboard logging, which triggered this line in the transformers library code. This would require a serialization of the Pyvene model's configs. Our model's configs contain types, not serializable by json.

[frankaging]

Marking this as close, and will track the progress in https://github.com/stanfordnlp/pyreft/issues/70

[Shigerello]

please try to add --report_to none and rerun. Thanks!

Below is another workaround in a Python code snippet. This approach prevents TensorBoard logging from attempting to serialize the Pyvene model's configs, which cannot be serialized due to the use of type.

from transformers.integrations.integration_utils import TensorBoardCallback
trainer.remove_callback(TensorBoardCallback)