allenai / RL4LMs

A modular RL library to fine-tune language models to human preferences
https://rl4lms.apps.allenai.org/
Apache License 2.0
2.13k stars 191 forks source link

NLPO Code Error and Query About gymnasium vs gym Usage #63

Open jinyilun718 opened 1 year ago

jinyilun718 commented 1 year ago

I hope this message finds you well. I am writing to report an issue I encountered in the NLPO project that you maintain on GitHub.

While executing the following block of code, in lines 278 of nlpo.py: if eval_env is not None and self.seed is not None: eval_env.seed(self.seed) eval_env = self._get_eval_env(eval_env) if not self._custom_logger:

I got:eval_env = self._get_eval_env(eval_env) AttributeError: 'NLPO' object has no attribute '_get_eval_env'.

I also found that no code related to '_get_eval_env'. in this repository, did i miss some key files? IF yes please let me know.

In addition to this, I have a query regarding library usage. I would like to know if it's possible to use 'gymnasium' in place of 'gym' within the NLPO project. If so, could you please guide on how to substitute all relevant 'gym' imports to 'gymnasium'?