RuntimeError for action shape

[lz1159435992]

I encountered issues with my previously working code after an update. I'm not sure what the source of the problem is, but there were no issues when using the version from a month ago. Can anyone tell me where I should focus my attention?

agent = PearlAgent(
                        policy_learner=SoftActorCritic(
                            state_dim=768,
                            action_space=action_space,
                            actor_hidden_dims=[768, 512, 128],
                            critic_hidden_dims=[768, 512, 128],
                            action_representation_module=action_representation_module,
                        ),
                        # history_summarization_module=StackingHistorySummarizationModule(
                        #     observation_dim=768,
                        #     action_dim=len(env.variables) + 1,
                        #     history_length=len(env.variables),
                        # ),
                        history_summarization_module=LSTMHistorySummarizationModule(
                            observation_dim=768,
                            action_dim=len(env.variables) + 1,
                            hidden_dim=768,
                            history_length=len(env.variables),  # 和完整结点数相同
                        ),
                        replay_buffer=FIFOOffPolicyReplayBuffer(10),
                        device_id=-1,
                    )
                    # 训练代理
                    info = online_learning(
                        agent=agent,
                        env=env,
                        number_of_episodes=number_of_episodes,
                        print_every_x_episodes=1,
                        record_period=record_period,
                        # learn_after_episode=True,
                    )

Traceback (most recent call last):
  File "/home/lz/PycharmProjects/Pearl/test_rl/test_group_gai_3_normal.py", line 181, in <module>
    test_group()
  File "/home/lz/PycharmProjects/Pearl/test_rl/test_group_gai_3_normal.py", line 150, in test_group
    info = online_learning(
  File "/home/lz/PycharmProjects/Pearl/test_rl/test_script/online_learning_break.py", line 115, in online_learning
    episode_info, episode_total_steps = run_episode(
  File "/home/lz/PycharmProjects/Pearl/test_rl/test_script/online_learning_break.py", line 285, in run_episode
    agent.observe(action_result)
  File "/home/lz/PycharmProjects/Pearl/pearl/pearl_agent.py", line 170, in observe
    new_subjective_state = self._update_subjective_state(action_result.observation)
  File "/home/lz/PycharmProjects/Pearl/pearl/pearl_agent.py", line 257, in _update_subjective_state
    return self.history_summarization_module.summarize_history(
  File "/home/lz/PycharmProjects/Pearl/pearl/history_summarization_modules/lstm_history_summarization_module.py", line 71, in summarize_history
    action = action.clone().detach().float().view((1, self.action_dim))
RuntimeError: shape '[1, 16]' is invalid for input of size 1

[rodrigodesalvobraz]

I encountered issues with my previously working code after an update. I'm not sure what the source of the problem is, but there were no issues when using the version from a month ago. Can anyone tell me where I should focus my attention?

Can you send us a link to a repo we can clone and run your example? The snippet in your message does not define several of the variables, such as action_space, env and so on. I could try to figure those out from your previous messages, but that would be time-consuming and there would be the risk of me not getting exactly the same thing you're running.

[lz1159435992]

https://github.com/lz1159435992/test_file.git This is my example code repository. Running the file test_group_gai_3_normal.py under /test_rl will cause problems. The test files are placed in the /files directory, and the file path may be problematic, requiring specific modifications. I apologize because my development is still in the early stages, and there is no documentation, and the code is quite messy. If you have any issues during the execution, please feel free to contact me at any time.

[rodrigodesalvobraz]

Hi. I finally got around to trying to reproduce this error, but then I hit line 10 in embedding_util.py which is from pearl.SMTimer.KNN_Predictor import Predictor, and get ModuleNotFoundError: No module named 'pearl.SMTimer'. However, pearl does not have an SMTimer subpackage, so I am unable to proceed.

Ideally, if you could gradually reduce the code until you create a minimum error-reproducing example involving only Pearl code and showing the error occurring, that would likely be much easier for us and might also be clarifying for you as well.