RuntimeError: invalid multinomial distribution (encountering probability entry < 0)

你好，代码运行以下测试的时候会报错: main.py --config=coma --env-config=one_step_matrix_game with save_model=True use_tensorboard=True save_model_interval=1000 t_max=50000 runner='episode' batch_size_run=1 use_cuda=False

报错信息： Traceback (most recent calls WITHOUT Sacred internals): File "D:/sby/RL/pymarl2/main.py", line 35, in my_main run_REGISTRY[_config['run']](_run, config, _log) File "D:\sby\RL\pymarl2\run\run.py", line 56, in run run_sequential(args=args, logger=logger) File "D:\sby\RL\pymarl2\run\run.py", line 181, in run_sequential episode_batch = runner.run(test_mode=False) File "D:\sby\RL\pymarl2\runners\episode_runner.py", line 70, in run actions = self.mac.select_actions(self.batch, t_ep=self.t, t_env=self.t_env, test_mode=test_mode) File "D:\sby\RL\pymarl2\controllers\basic_controller.py", line 23, in select_actions t_env, test_mode=test_mode) File "D:\sby\RL\pymarl2\components\action_selectors.py", line 105, in select_action picked_actions = Categorical(masked_policies).sample().long() File "D:\Anaconda3\lib\site-packages\torch\distributions\categorical.py", line 107, in sample samples_2d = torch.multinomial(probs_2d, sample_shape.numel(), True).T RuntimeError: invalid multinomial distribution (encountering probability entry < 0)

我找了下问题，应该是rnn_agent.py中，x = F.relu(self.fc1(inputs.view(-1, e)), inplace=True)，多次迭代后，梯度累积，导致梯度爆炸，从而输出存在nan. 你看能否解决一下，谢谢。我的环境是win10, pytorch1.x

谢谢同学提出的bug 我们没有充分测试试过COMA哦，只有论文中提到的算法都仔细测试了。 ———————————————————— 刚才看了下，应该已经修复了，entropy loss 写错了。

你好，代码运行以下测试的时候会报错: main.py --config=coma --env-config=one_step_matrix_game with save_model=True use_tensorboard=True save_model_interval=1000 t_max=50000 runner='episode' batch_size_run=1 use_cuda=False

报错信息： Traceback (most recent calls WITHOUT Sacred internals): File "D:/sby/RL/pymarl2/main.py", line 35, in my_main run_REGISTRY[_config['run']](_run, config, _log) File "D:\sby\RL\pymarl2\run\run.py", line 56, in run run_sequential(args=args, logger=logger) File "D:\sby\RL\pymarl2\run\run.py", line 181, in run_sequential episode_batch = runner.run(test_mode=False) File "D:\sby\RL\pymarl2\runners\episode_runner.py", line 70, in run actions = self.mac.select_actions(self.batch, t_ep=self.t, t_env=self.t_env, test_mode=test_mode) File "D:\sby\RL\pymarl2\controllers\basic_controller.py", line 23, in select_actions t_env, test_mode=test_mode) File "D:\sby\RL\pymarl2\components\action_selectors.py", line 105, in select_action picked_actions = Categorical(masked_policies).sample().long() File "D:\Anaconda3\lib\site-packages\torch\distributions\categorical.py", line 107, in sample samples_2d = torch.multinomial(probs_2d, sample_shape.numel(), True).T RuntimeError: invalid multinomial distribution (encountering probability entry < 0)

我找了下问题，应该是rnn_agent.py中，x = F.relu(self.fc1(inputs.view(-1, e)), inplace=True)，多次迭代后，梯度累积，导致梯度爆炸，从而输出存在nan. 你看能否解决一下，谢谢。我的环境是win10, pytorch1.x

请问你是如何在win10 环境下跑通代码的， YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. config_dict = yaml.load(f) 我出现这样的错误可能是环境文件夹位置造成的，按照readme，应该是pymarl下有3rdparty，再安装StarCraftII？还请指导一下，谢谢

你好，代码运行以下测试的时候会报错: main.py --config=coma --env-config=one_step_matrix_game with save_model=True use_tensorboard=True save_model_interval=1000 t_max=50000 runner='episode' batch_size_run=1 use_cuda=False 报错信息： Traceback (most recent calls WITHOUT Sacred internals): File "D:/sby/RL/pymarl2/main.py", line 35, in my_main run_REGISTRY[_config['run']](_run, config, _log) File "D:\sby\RL\pymarl2\run\run.py", line 56, in run run_sequential(args=args, logger=logger) File "D:\sby\RL\pymarl2\run\run.py", line 181, in run_sequential episode_batch = runner.run(test_mode=False) File "D:\sby\RL\pymarl2\runners\episode_runner.py", line 70, in run actions = self.mac.select_actions(self.batch, t_ep=self.t, t_env=self.t_env, test_mode=test_mode) File "D:\sby\RL\pymarl2\controllers\basic_controller.py", line 23, in select_actions t_env, test_mode=test_mode) File "D:\sby\RL\pymarl2\components\action_selectors.py", line 105, in select_action picked_actions = Categorical(masked_policies).sample().long() File "D:\Anaconda3\lib\site-packages\torch\distributions\categorical.py", line 107, in sample samples_2d = torch.multinomial(probs_2d, sample_shape.numel(), True).T RuntimeError: invalid multinomial distribution (encountering probability entry < 0) 我找了下问题，应该是rnn_agent.py中，x = F.relu(self.fc1(inputs.view(-1, e)), inplace=True)，多次迭代后，梯度累积，导致梯度爆炸，从而输出存在nan. 你看能否解决一下，谢谢。我的环境是win10, pytorch1.x

请问你是如何在win10 环境下跑通代码的， YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. config_dict = yaml.load(f) 我出现这样的错误可能是环境文件夹位置造成的，按照readme，应该是pymarl下有3rdparty，再安装StarCraftII？还请指导一下，谢谢

这个warning的解决方案是：config_dict = yaml.load(f, Loader=yaml.FullLoader) 在win10下运行需要注释掉run.py中的os._exit(os.EX_OK)，因为 os.EX_OK and os.EX_NOINPUT constants are not available on Windows

如果不能解决你的问题，请给一下完整的报错信息，我看看。

你好，代码运行以下测试的时候会报错: main.py --config=coma --env-config=one_step_matrix_game with save_model=True use_tensorboard=True save_model_interval=1000 t_max=50000 runner='episode' batch_size_run=1 use_cuda=False 报错信息： Traceback (most recent calls WITHOUT Sacred internals): File "D:/sby/RL/pymarl2/main.py", line 35, in my_main run_REGISTRY[_config['run']](_run, config, _log) File "D:\sby\RL\pymarl2\run\run.py", line 56, in run run_sequential(args=args, logger=logger) File "D:\sby\RL\pymarl2\run\run.py", line 181, in run_sequential episode_batch = runner.run(test_mode=False) File "D:\sby\RL\pymarl2\runners\episode_runner.py", line 70, in run actions = self.mac.select_actions(self.batch, t_ep=self.t, t_env=self.t_env, test_mode=test_mode) File "D:\sby\RL\pymarl2\controllers\basic_controller.py", line 23, in select_actions t_env, test_mode=test_mode) File "D:\sby\RL\pymarl2\components\action_selectors.py", line 105, in select_action picked_actions = Categorical(masked_policies).sample().long() File "D:\Anaconda3\lib\site-packages\torch\distributions\categorical.py", line 107, in sample samples_2d = torch.multinomial(probs_2d, sample_shape.numel(), True).T RuntimeError: invalid multinomial distribution (encountering probability entry < 0) 我找了下问题，应该是rnn_agent.py中，x = F.relu(self.fc1(inputs.view(-1, e)), inplace=True)，多次迭代后，梯度累积，导致梯度爆炸，从而输出存在nan. 你看能否解决一下，谢谢。我的环境是win10, pytorch1.x

请问你是如何在win10 环境下跑通代码的， YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. config_dict = yaml.load(f) 我出现这样的错误可能是环境文件夹位置造成的，按照readme，应该是pymarl下有3rdparty，再安装StarCraftII？还请指导一下，谢谢

这个warning的解决方案是：config_dict = yaml.load(f, Loader=yaml.FullLoader) 在win10下运行需要注释掉run.py中的os._exit(os.EX_OK)，因为 os.EX_OK and os.EX_NOINPUT constants are not available on Windows

如果不能解决你的问题，请给一下完整的报错信息，我看看。

PS D:\star\pymarl2-master> & 'C:\Users\lenovo\anaconda3\envs\pymarl\python.exe' 'c:\Users\lenovo.vscode\extensions\ms-python.python-2021.8.1105858891\pythonFiles\lib\python\debugpy\launcher' '53279' '--' 'd:\star\pymarl2-master\src\main.py' pygame 2.0.0 (SDL 2.0.12, python 3.7.10) Hello from the pygame community. https://www.pygame.org/contribute.html [DEBUG 16:41:31] git.cmd Popen(['git', 'version'], cwd=D:\star\pymarl2-master, universal_newlines=False, shell=None, istream=None) [DEBUG 16:41:31] git.cmd Popen(['git', 'version'], cwd=D:\star\pymarl2-master, universal_newlines=False, shell=None, istream=None) [DEBUG 16:41:31] git.util Failed checking if running in CYGWIN due to: FileNotFoundError(2, '系统找不到指定的文件。', None, 2, None) Backend TkAgg is interactive backend. Turning interactive mode on. Traceback (most recent call last): File "C:\Users\lenovo\anaconda3\envs\pymarl\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "C:\Users\lenovo\anaconda3\envs\pymarl\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "c:\Users\lenovo.vscode\extensions\ms-python.python-2021.8.1105858891\pythonFiles\lib\python\debugpy__main.py", line 45, in cli.main() File "c:\Users\lenovo.vscode\extensions\ms-python.python-2021.8.1105858891\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 444, in main run() File "c:\Users\lenovo.vscode\extensions\ms-python.python-2021.8.1105858891\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 285, in run_file runpy.run_path(target_as_str, run_name=compat.force_str("main__")) File "C:\Users\lenovo\anaconda3\envs\pymarl\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "C:\Users\lenovo\anaconda3\envs\pymarl\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "C:\Users\lenovo\anaconda3\envs\pymarl\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "d:\star\pymarl2-master\src\main.py", line 96, in config_dict = recursive_dict_update(config_dict, env_config) File "d:\star\pymarl2-master\src\main.py", line 55, in recursive_dict_update for k, v in u.items(): AttributeError: 'NoneType' object has no attribute 'items' PS D:\star\pymarl2-master> conda activate pymarl conda : 无法将“conda”项识别为 cmdlet、函数、脚本文件或可运行程序的名称。请检查名称的拼写，如果包括路径，请确保路径正确，然后再试一次。所在位置行:1 字符: 1

conda activate pymarl


+ CategoryInfo          : ObjectNotFound: (conda:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException

Thanks.

你好，代码运行以下测试的时候会报错: main.py --config=coma --env-config=one_step_matrix_game with save_model=True use_tensorboard=True save_model_interval=1000 t_max=50000 runner='episode' batch_size_run=1 use_cuda=False 报错信息： Traceback (most recent calls WITHOUT Sacred internals): File "D:/sby/RL/pymarl2/main.py", line 35, in my_main run_REGISTRY[_config['run']](_run, config, _log) File "D:\sby\RL\pymarl2\run\run.py", line 56, in run run_sequential(args=args, logger=logger) File "D:\sby\RL\pymarl2\run\run.py", line 181, in run_sequential episode_batch = runner.run(test_mode=False) File "D:\sby\RL\pymarl2\runners\episode_runner.py", line 70, in run actions = self.mac.select_actions(self.batch, t_ep=self.t, t_env=self.t_env, test_mode=test_mode) File "D:\sby\RL\pymarl2\controllers\basic_controller.py", line 23, in select_actions t_env, test_mode=test_mode) File "D:\sby\RL\pymarl2\components\action_selectors.py", line 105, in select_action picked_actions = Categorical(masked_policies).sample().long() File "D:\Anaconda3\lib\site-packages\torch\distributions\categorical.py", line 107, in sample samples_2d = torch.multinomial(probs_2d, sample_shape.numel(), True).T RuntimeError: invalid multinomial distribution (encountering probability entry < 0) 我找了下问题，应该是rnn_agent.py中，x = F.relu(self.fc1(inputs.view(-1, e)), inplace=True)，多次迭代后，梯度累积，导致梯度爆炸，从而输出存在nan. 你看能否解决一下，谢谢。我的环境是win10, pytorch1.x

请问你是如何在win10 环境下跑通代码的， YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. config_dict = yaml.load(f) 我出现这样的错误可能是环境文件夹位置造成的，按照readme，应该是pymarl下有3rdparty，再安装StarCraftII？还请指导一下，谢谢

这个warning的解决方案是：config_dict = yaml.load(f, Loader=yaml.FullLoader) 在win10下运行需要注释掉run.py中的os._exit(os.EX_OK)，因为 os.EX_OK and os.EX_NOINPUT constants are not available on Windows 如果不能解决你的问题，请给一下完整的报错信息，我看看。

PS D:\star\pymarl2-master> & 'C:\Users\lenovo\anaconda3\envs\pymarl\python.exe' 'c:\Users\lenovo.vscode\extensions\ms-python.python-2021.8.1105858891\pythonFiles\lib\python\debugpy\launcher' '53279' '--' 'd:\star\pymarl2-master\src\main.py' pygame 2.0.0 (SDL 2.0.12, python 3.7.10) Hello from the pygame community. https://www.pygame.org/contribute.html [DEBUG 16:41:31] git.cmd Popen(['git', 'version'], cwd=D:\star\pymarl2-master, universal_newlines=False, shell=None, istream=None) [DEBUG 16:41:31] git.cmd Popen(['git', 'version'], cwd=D:\star\pymarl2-master, universal_newlines=False, shell=None, istream=None) [DEBUG 16:41:31] git.util Failed checking if running in CYGWIN due to: FileNotFoundError(2, '系统找不到指定的文件。', None, 2, None) Backend TkAgg is interactive backend. Turning interactive mode on. Traceback (most recent call last): File "C:\Users\lenovo\anaconda3\envs\pymarl\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "C:\Users\lenovo\anaconda3\envs\pymarl\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "c:\Users\lenovo.vscode\extensions\ms-python.python-2021.8.1105858891\pythonFiles\lib\python\debugpymain.py", line 45, in cli.main() File "c:\Users\lenovo.vscode\extensions\ms-python.python-2021.8.1105858891\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 444, in main run() File "c:\Users\lenovo.vscode\extensions\ms-python.python-2021.8.1105858891\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 285, in run_file runpy.run_path(target_as_str, run_name=compat.force_str("main")) File "C:\Users\lenovo\anaconda3\envs\pymarl\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "C:\Users\lenovo\anaconda3\envs\pymarl\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "C:\Users\lenovo\anaconda3\envs\pymarl\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "d:\star\pymarl2-master\src\main.py", line 96, in config_dict = recursive_dict_update(config_dict, env_config) File "d:\star\pymarl2-master\src\main.py", line 55, in recursive_dict_update for k, v in u.items(): AttributeError: 'NoneType' object has no attribute 'items' PS D:\star\pymarl2-master> conda activate pymarl conda : 无法将“conda”项识别为 cmdlet、函数、脚本文件或可运行程序的名称。请检查名称的拼写，如果包括路径，请确保路径正确，然后再试一次。所在位置行:1 字符: 1
conda activate pymarl
+ CategoryInfo          : ObjectNotFound: (conda:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
Thanks.

conda没有添加到环境变量上一个报错：u 是NoneType，所以没有属性items AttributeError: 'NoneType' object has no attribute 'items'

hijkzzz / pymarl2

RuntimeError: invalid multinomial distribution (encountering probability entry < 0) #1