When I run the bash script for step 1 and step 2, the following issues happens (No module named 'utils.model.token_reward_model'):
/home/jeeves/.conda/envs/tlcr/lib/python3.10/site-packages/transformers/deepspeed.py:23:
FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
warnings.warn(
Traceback (most recent call last):
File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/step2_reward_model_finetuning/main.py", line 30, in
from utils.model.model_utils import create_critic_model, create_token_critic_model
File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/utils/model/model_utils.py", line 17, in
/home/jeeves/.conda/envs/tlcr/lib/python3.10/site-packages/transformers/deepspeed.py:23: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
warnings.warn(
from .token_reward_model import TokenRewardModelTraceback (most recent call last):
File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/step2_reward_model_finetuning/main.py", line 30, in
ModuleNotFoundError: No module named 'utils.model.token_reward_model'
from utils.model.model_utils import create_critic_model, create_token_critic_model
File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/utils/model/model_utils.py", line 17, in
from .token_reward_model import TokenRewardModel
ModuleNotFoundError: No module named 'utils.model.token_reward_model'
/home/jeeves/.conda/envs/tlcr/lib/python3.10/site-packages/transformers/deepspeed.py:23: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
warnings.warn(
/home/jeeves/.conda/envs/tlcr/lib/python3.10/site-packages/transformers/deepspeed.py:23: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
warnings.warn(
Traceback (most recent call last):
File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/step2_reward_model_finetuning/main.py", line 30, in
Traceback (most recent call last):
File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/step2_reward_model_finetuning/main.py", line 30, in
from utils.model.model_utils import create_critic_model, create_token_critic_model
File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/utils/model/model_utils.py", line 17, in
from utils.model.model_utils import create_critic_model, create_token_critic_model
File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/utils/model/model_utils.py", line 17, in
from .token_reward_model import TokenRewardModel
ModuleNotFoundError: No module named 'utils.model.token_reward_model'
from .token_reward_model import TokenRewardModel
ModuleNotFoundError: No module named 'utils.model.token_reward_model'
When I run the bash script for step 1 and step 2, the following issues happens (No module named 'utils.model.token_reward_model'):
/home/jeeves/.conda/envs/tlcr/lib/python3.10/site-packages/transformers/deepspeed.py:23: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations warnings.warn( Traceback (most recent call last): File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/step2_reward_model_finetuning/main.py", line 30, in
from utils.model.model_utils import create_critic_model, create_token_critic_model
File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/utils/model/model_utils.py", line 17, in
/home/jeeves/.conda/envs/tlcr/lib/python3.10/site-packages/transformers/deepspeed.py:23: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
warnings.warn(
from .token_reward_model import TokenRewardModelTraceback (most recent call last):
File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/step2_reward_model_finetuning/main.py", line 30, in
ModuleNotFoundError: No module named 'utils.model.token_reward_model'
from utils.model.model_utils import create_critic_model, create_token_critic_model
File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/utils/model/model_utils.py", line 17, in
from .token_reward_model import TokenRewardModel
ModuleNotFoundError: No module named 'utils.model.token_reward_model'
/home/jeeves/.conda/envs/tlcr/lib/python3.10/site-packages/transformers/deepspeed.py:23: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
warnings.warn(
/home/jeeves/.conda/envs/tlcr/lib/python3.10/site-packages/transformers/deepspeed.py:23: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
warnings.warn(
Traceback (most recent call last):
File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/step2_reward_model_finetuning/main.py", line 30, in
Traceback (most recent call last):
File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/step2_reward_model_finetuning/main.py", line 30, in
from utils.model.model_utils import create_critic_model, create_token_critic_model
File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/utils/model/model_utils.py", line 17, in
from utils.model.model_utils import create_critic_model, create_token_critic_model
File "/data/lzy/rlhf/dense_rlhf/RLHF-TLCR-master/utils/model/model_utils.py", line 17, in
from .token_reward_model import TokenRewardModel
ModuleNotFoundError: No module named 'utils.model.token_reward_model'
from .token_reward_model import TokenRewardModel
ModuleNotFoundError: No module named 'utils.model.token_reward_model'