Closed nimisha-stellosys closed 2 years ago
The reason for this error is that the ACKTR implementation we borrowed from https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail is not robust enough. Three suggestions to solve this problem:
Thank you I will try with these parameters.
The reason for this error is that the ACKTR implementation we borrowed from https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail is not robust enough. Three suggestions to solve this problem:
- Increase the value of the mask (in config.py) during training.
- Modify the hyperparameters in the algorithm so that the training algorithm can avoid this convergence error.
- This convergence problem only appears in the implementation of ACKTR, you can try to change the training algorithm to a2c or other reinforcement learning algorithms.
Hi @alexfrom0815 If I use a2c instead of ACKTR will there be any performance downside? Or its ok to use it
The reason for this error is that the ACKTR implementation we borrowed from https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail is not robust enough. Three suggestions to solve this problem:
- Increase the value of the mask (in config.py) during training.
- Modify the hyperparameters in the algorithm so that the training algorithm can avoid this convergence error.
- This convergence problem only appears in the implementation of ACKTR, you can try to change the training algorithm to a2c or other reinforcement learning algorithms.
Hi @alexfrom0815 If I use a2c instead of ACKTR will there be any performance downside? Or its ok to use it
Unfortunately, the a2c algorithm is indeed much worse than ACKTR. We are currently looking for better alternatives to the ACKTR algorithm. We encourage you to try some other algorithmic hyperparameters to avoid this ACKTR error (random seed, mask value, etc.).
Another tip is that although we cannot determine the cause of this error, from experience, when the error appears, the performance of the algorithm is generally close to the highest value, so its existence may not affect the final effect of the algorithm in a certain sense.
In addition, we have completed the implementation of a new online BPP algorithm, which is more stable than the current algorithm and can work at arbitrary resolution. Due to the double-blind protocol of the paper review, we will publish the new algorithm code after our new work is accepted.
The reason for this error is that the ACKTR implementation we borrowed from https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail is not robust enough. Three suggestions to solve this problem:
1. Increase the value of the mask (in config.py) during training. 2. Modify the hyperparameters in the algorithm so that the training algorithm can avoid this convergence error. 3. This convergence problem only appears in the implementation of ACKTR, you can try to change the training algorithm to a2c or other reinforcement learning algorithms.
where is config.py? I did not find it.
Traceback (most recent call last): File "main.py", line 258, in
main(args)
File "main.py", line 42, in main
train_model()
File "main.py", line 209, in train_model
value_loss, action_loss, dist_entropy, prob_loss, graph_loss = agent.update(rollouts)
File "D:\Online-3D-BPP-DRL-main\acktr\algo\acktr_pipeline.py", line 98, in update
self.optimizer.step()
File "C:\Users\Sty\anaconda3\envs\TF-GPU\lib\site-packages\torch\optim\optimizer.py", line 88, in wrapper
return func(*args, **kwargs)
File "D:\Online-3D-BPP-DRL-main\acktr\algo\kfac.py", line 215, in step
self.d_a[m], self.Q_a[m] = torch.symeig(
RuntimeError: symeig_cuda: the algorithm failed to converge; 1001 off-diagonal elements of an intermediate tridiagonal form did not converge to zero.