Fail to load the provided Math-psa reward model

[YanSong97]

System Info

Ubuntu, Python 3.10, A100 GPU

Who can help?

@YanSong97

Information

• [X] The official example scripts
• [ ] My own modified scripts

Tasks

• [X] An officially supported task in the codebase (such as scrips/, ...)
• [ ] My own task or dataset (give details below)

Reproduction

1. change VALUE_MODEL_PATH variable to path where checkpoint-2127 is located;
2. Run create_service_qwen2.5_math_vllm.sh
3. Encounter error message:

OSError: Incorrect path_or_model_id: '../../models/Qwen/Qwen2.5-Math-7B-Instruct/'. Please provide either the path to a local folder or the repo_id of a model on the Hub.

Expected behavior

The RM is normally loaded.

[YanSong97]

This is because the provided PRM Math-psa is no more than a Lora adapter. In checkpoint-2127/adapter_config.json you can see "base_model_name_or_path": "../../models/Qwen/Qwen2.5-Math-7B-Instruct/" which represents where the base model is located. Change this to your own directory and you will be able to load it successfully.