Inference error after fine-tuning the model

[Neo9061]

Hello, i have followed the distributed_train.py and finished training a florence base ft model. However, I tried to use it for inference. I hit error during model loading stage using the following code, same code used in training.

model = AutoModelForCausalLM.from_pretrained(fine_tuned_path, torch_dtype=torch_dtype, trust_remote_code=True).to("cuda:0")

The error is that following.

  File "/opt/conda/envs/myenv/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3832, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
  File "/home/ubuntu/.cache/huggingface/modules/transformers_modules/epoch_10/modeling_florence2.py", line 2535, in __init__
    assert config.vision_config.model_type == 'davit', 'only DaViT is supported for now'
AssertionError: only DaViT is supported for now

I looked at the checkpoints that are saved. Its architecture config.json is a different from the original downloaded config.json.

More importantly, I tried to modified the scripts modeling_florence2.py to remove the assertion check. However, it seems the code always try to look at the ./cache dir with /home/ubuntu/.cache/huggingface/modules/transformers_modules/epoch_10/modeling_florence2.py

Do you know what might happen? THX

[Neo9061]

Figured out. When we use trust_remote=True, we need modfiy the script path in the config.json to point to the local