RuntimeError: Can't call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead.

[hoang6k]

Hi, I tried to run the l2l code with: python main.py --train --problem bbob --difficulty easy --train_agent L2L_Agent --train_optimizer L2L_Optimizer --max_learning_step 150000 but it returned error:

start training: 20240315T134833_bbob_easy_10D Training L2L_Agent Epoch 0: 0%| | 0/18 [00:00<?, ?it/s] Traceback (most recent call last): File "D:\git-repos\MetaBox\src\main.py", line 24, in trainer.train() File "D:\git-repos\MetaBox\src\trainer.py", line 151, in train exceed_max_ls, pbar_info_train = self.agent.train_episode(env) # pbar_info -> dict File "D:\git-repos\MetaBox\src\agent\l2l_agent.py", line 49, in trainepisode y,,_=env.step(x) File "D:\git-repos\MetaBox\src\environment\basic_environment.py", line 22, in step return self.optimizer.update(action, self.problem) File "D:\git-repos\MetaBox\src\optimizer\l2l_optimizer.py", line 41, in update y=problem.eval(x)-problem.optimum File "D:\git-repos\MetaBox\src\problem\basic_problem.py", line 21, in eval x = np.array(x) File "C:\Users\minhh\miniconda3\envs\py39\lib\site-packages\torch_tensor.py", line 643, in array return self.numpy() RuntimeError: Can't call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead.

The line number might differ a little bit since I added some comment code. According to the suggestion, I replace x = np.array(x) by x = x.detach().numpy() but it returned another error:

start training: 20240315T135459_bbob_easy_10D Training L2L_Agent Epoch 0: 0%| | 0/18 [00:00<?, ?it/s] Traceback (most recent call last): File "D:\git-repos\MetaBox\src\main.py", line 24, in trainer.train() File "D:\git-repos\MetaBox\src\trainer.py", line 151, in train exceed_max_ls, pbar_info_train = self.agent.train_episode(env) # pbar_info -> dict File "D:\git-repos\MetaBox\src\agent\l2l_agent.py", line 65, in train_episode loss.mean().backward() AttributeError: 'numpy.float64' object has no attribute 'backward'

What did I do wrong here?

[JC-Chen1]

According to the running command you have provided, it seems that the --problem setting is set wrongly to bbob which should be bbob-torch for RNN-OI style approach. ...-torch dataset is designed for gradient-required scenario that will be operated under Pytorch Tensor environment that will record gradient automatically.

code reference: https://github.com/GMC-DRL/MetaBox/blob/bb1177df130c4c0eb80892758c82cc83a109425c/src/config.py#L56

[JC-Chen1]

Thank you for raising this discussion, we will update the user guide doc to elaborate the particularity of RNN-OI usage to make MetaBox work more smoothly.

[hoang6k]

Thank you for getting back to me promptly, I appreciate it. The repository is really convenient for trying things out, nicely done!