search

Swopper050 / foundations-drl

Implementing the base algorithms of Deep Reinforcement Learning in Python
18 stars 1 forks source link

Having trouble running simple REINFORCE example #25

Closed aveshd closed 11 months ago

aveshd commented 1 year ago

Hi, Thank you so much for writing an excellent book and provided code. I am just learning the ropes here. I tried running simple REINFORCE algorithm as suggested in the github page. There are already bunch of issues with the versions used in requirements.txt. Especially Pytorch version which seem quite old and need python version 3.6. Visual code no longer supports the version 3.6 for debugging

python train_agent.py --env-name CartPole-v0 --algorithm reinforce --save-name cartpole_v0

Anyway I was able to install all the required versions suggested in requiremets.txt However I got the following error Traceback (most recent call last): File "train_agent.py", line 85, in <module> main(args) File "train_agent.py", line 51, in main render=args.render, File "/Users/user1/Documents/rl/foundations-drl/algorithms/reinforce/reinforce_trainer.py", line 56, in train_agent action = agent.act(obs, deterministic=False) File "/Users/user1/Documents/rl/foundations-drl/algorithms/reinforce/discrete_reinforce_agent.py", line 99, in act logits = self.forward(torch.from_numpy(np.asarray(observation[0]))) File "/Users/user1/Documents/rl/foundations-drl/algorithms/reinforce/discrete_reinforce_agent.py", line 84, in forward return self.net(X) File "/Users/user1/anaconda3/envs/rl2/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/Users/user1/anaconda3/envs/rl2/lib/python3.6/site-packages/torch/nn/modules/container.py", line 119, in forward input = module(input) File "/Users/user1/anaconda3/envs/rl2/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/Users/user1/anaconda3/envs/rl2/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 94, in forward return F.linear(input, self.weight, self.bias) File "/Users/user1/anaconda3/envs/rl2/lib/python3.6/site-packages/torch/nn/functional.py", line 1753, in linear return torch._C._nn.linear(input, weight, bias) RuntimeError: both arguments to matmul need to be at least 1D, but they are 0D and 2D

Please let me know if you have updated version of the code regards,

Swopper050 commented 1 year ago

Hi @aveshd, thanks for your issue :) First of all, I did not write the book! It was written by Laura Graesser and Wah Loon Keng.

As for your errors, I will see if I can reproduce your error and submit a fix soon!

Swopper050 commented 11 months ago

@aveshd Very late, but in case you're still interested, it should work now.