search

EsYoon7 / RLHF-TLCR

[ACL'24 Findings] Official code for "TLCR: Token-Level Continuous Reward for Fine-grained Reinforcement Learning from Human Feedback"
4 stars 0 forks source link

ModuleNotFoundError: No module named 'utils.model.token_reward_model' #2

Open Zeyuan-Liu opened 1 month ago

Zeyuan-Liu commented 1 month ago

When I run the bash script for step 1 and step 2, the following issues happens (No module named 'utils.model.token_reward_model'):

/home/jeeves/.conda/envs/tlcr/lib/python3.10/site-packages/transformers/deepspeed.py:23: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations warnings.warn( Traceback (most recent call last): File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/step2_reward_model_finetuning/main.py", line 30, in from utils.model.model_utils import create_critic_model, create_token_critic_model File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/utils/model/model_utils.py", line 17, in /home/jeeves/.conda/envs/tlcr/lib/python3.10/site-packages/transformers/deepspeed.py:23: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations warnings.warn( from .token_reward_model import TokenRewardModelTraceback (most recent call last):

File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/step2_reward_model_finetuning/main.py", line 30, in ModuleNotFoundError: No module named 'utils.model.token_reward_model' from utils.model.model_utils import create_critic_model, create_token_critic_model File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/utils/model/model_utils.py", line 17, in from .token_reward_model import TokenRewardModel ModuleNotFoundError: No module named 'utils.model.token_reward_model' /home/jeeves/.conda/envs/tlcr/lib/python3.10/site-packages/transformers/deepspeed.py:23: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations warnings.warn( /home/jeeves/.conda/envs/tlcr/lib/python3.10/site-packages/transformers/deepspeed.py:23: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations warnings.warn( Traceback (most recent call last): File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/step2_reward_model_finetuning/main.py", line 30, in Traceback (most recent call last): File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/step2_reward_model_finetuning/main.py", line 30, in from utils.model.model_utils import create_critic_model, create_token_critic_model File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/utils/model/model_utils.py", line 17, in from utils.model.model_utils import create_critic_model, create_token_critic_model File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/utils/model/model_utils.py", line 17, in from .token_reward_model import TokenRewardModel ModuleNotFoundError: No module named 'utils.model.token_reward_model' from .token_reward_model import TokenRewardModel ModuleNotFoundError: No module named 'utils.model.token_reward_model'

EsYoon7 commented 1 month ago

Sorry for late reply, I updated the code if you still have the issue please let me know