search

hakuhodo-technologies / scope-rl

SCOPE-RL: A python library for offline reinforcement learning, off-policy evaluation, and selection
https://scope-rl.readthedocs.io/en/latest/
Apache License 2.0
106 stars 10 forks source link

Fixing errors with _obtain_steps function #27

Open alexofficial opened 5 months ago

alexofficial commented 5 months ago

Hi,

I tried to collect the offline dataset of an online trained model using the _obtain_steps function. I found an error there, in lines 694-697, where two parenthesis are missing inside the file: scope-rl/scope-rl/dataset/synthetic.py image

The solution is:

actions = np.zeros((n_trajectories * step_per_trajectory, self.action_dim))
action_probs = np.zeros((n_trajectories * step_per_trajectory,self.action_dim))

Secondly, there is a variable rollout_lengths (line 704), which causes an error in line 723 within the for loop. However, since there are no examples with _obtain_steps, I could not compare this. A quick solution was to simply change it from: for rollout_step inrollout_lengths[i]:to forrollout_step in range(rollout_lengths[i]):`

Note: I have now successfully trained an offline RL model with the _obtain_steps function. Let me know if I can contribute in any way :)

aiueola commented 4 months ago

@alexofficial

Thank you for reaching out with the issues and solutions! The codes (both lines 694-697 and line 723) should definitely be fixed as you mentioned. Could you make a PR with the suggested update? We appreciate your help in improving the software.