search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.19k stars 4.16k forks source link

List index out of range error at end of curriculum #4273

Closed HPRIOR closed 4 years ago

HPRIOR commented 4 years ago

Agent finished a curriculum, after the model had been saved the terminal showed 'IndexError: list index out of range'

2020-07-25 20:52:17 INFO [stats.py:129] Hyperparameters for behavior name Spy:
        trainer_type:   ppo
        hyperparameters:
          batch_size:   128
          buffer_size:  2048
          learning_rate:        0.0003
          beta: 0.01
          epsilon:      0.2
          lambd:        0.95
          num_epoch:    3
          learning_rate_schedule:       linear
        network_settings:
          normalize:    False
          hidden_units: 512
          num_layers:   2
          vis_encode_type:      simple
          memory:       None
        reward_signals:
          extrinsic:
            gamma:      0.99
            strength:   1.0
          curiosity:
            gamma:      0.99
            strength:   0.2
            encoding_size:      256
            learning_rate:      0.0003
        init_path:      None
        keep_checkpoints:       5
        checkpoint_interval:    500000
        max_steps:      10000000
        time_horizon:   128
        summary_freq:   30000
        threaded:       True
        self_play:      None
        behavioral_cloning:     None

2020-07-25 21:13:42 INFO [stats.py:101] Spy: Step: 1110000. Time Elapsed: 1292.975 s Mean Reward: 0.870. Std of Rewar
d: 0.282. Training.

2020-07-25 21:14:21 INFO [stats.py:101] Spy: Step: 1140000. Time Elapsed: 1331.635 s Mean Reward: 0.920. Std of Rewar
d: 0.157. Training.

2020-07-25 21:14:24 INFO [trainer_controller.py:76] Saved Model
2020-07-25 21:14:24 INFO [model_serialization.py:203] List of nodes to export for brain :Spy
2020-07-25 21:14:24 INFO [model_serialization.py:205]   is_continuous_control
2020-07-25 21:14:24 INFO [model_serialization.py:205]   trainer_major_version
2020-07-25 21:14:24 INFO [model_serialization.py:205]   trainer_minor_version
2020-07-25 21:14:24 INFO [model_serialization.py:205]   trainer_patch_version
2020-07-25 21:14:24 INFO [model_serialization.py:205]   version_number
2020-07-25 21:14:24 INFO [model_serialization.py:205]   memory_size
2020-07-25 21:14:24 INFO [model_serialization.py:205]   action_output_shape
2020-07-25 21:14:24 INFO [model_serialization.py:205]   action
Converting results\spy_test_curriculum_mem_obvs_normalised_02\Spy/frozen_graph_def.pb to results\spy_test_curriculum_mem_obvs_normalised_02\Spy.nn
GLOBALS: 'is_continuous_control', 'trainer_major_version', 'trainer_minor_version', 'trainer_patch_version', 'version_number', 'memory_size', 'action_output_shape'
IN: 'vector_observation': [-1, 1, 1, 43] => 'policy/main_graph_0/hidden_0/BiasAdd'
IN: 'action_masks': [-1, 1, 1, 5] => 'policy_1/strided_slice'
OUT: 'policy_1/concat_2/concat', 'action'
DONE: wrote results\spy_test_curriculum_mem_obvs_normalised_02\Spy.nn file.
2020-07-25 21:14:24 INFO [model_serialization.py:83] Exported results\spy_test_curriculum_mem_obvs_normalised_02\Spy.nn file
Traceback (most recent call last):
Converting results\spy_test_curriculum_mem_obvs_normalised_02\Spy/frozen_graph_def.pb to results\spy_test_curriculum_
mem_obvs_normalised_02\Spy.nn
GLOBALS: 'is_continuous_control', 'trainer_major_version', 'trainer_minor_version', 'trainer_patch_version', 'version
_number', 'memory_size', 'action_output_shape'
IN: 'vector_observation': [-1, 1, 1, 43] => 'policy/main_graph_0/hidden_0/BiasAdd'
IN: 'action_masks': [-1, 1, 1, 5] => 'policy_1/strided_slice'
OUT: 'policy_1/concat_2/concat', 'action'
DONE: wrote results\spy_test_curriculum_mem_obvs_normalised_02\Spy.nn file.
2020-07-25 21:14:24 INFO [model_serialization.py:83] Exported results\spy_test_curriculum_mem_obvs_normalised_02\Spy.nn file
Traceback (most recent call last):
  File "c:\users\user\appdata\local\programs\python\python38\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\users\user\appdata\local\programs\python\python38\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\Scripts\mlagents-learn.exe\__main__.py", line 9, in <module>
  File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\mlagents\trainers\learn.py", line 283, in main
    run_cli(parse_command_line())
  File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\mlagents\trainers\learn.py", line 279, in run_cli
    run_training(run_seed, options)
  File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\mlagents\trainers\learn.py", line 158, in run_training
    tc.start_learning(env_manager)
  File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\mlagents_envs\timers.py", line 305, in wrapped
    return func(*args, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\mlagents\trainers\trainer_controller.py", line 185, in start_learning
    self.reset_env_if_ready(env_manager)
  File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\mlagents\trainers\trainer_controller.py", line 224, in reset_env_if_ready
    updated, param_must_reset = self.param_manager.update_lessons(
  File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\mlagents\trainers\environment_parameter_manager.py", line 149, in update_lessons
    new_lesson_name = settings.curriculum[lesson_num + 1].name
IndexError: list index out of range
awjuliani commented 4 years ago

Hi @HPRIOR

Can you share your curriculum file?

HPRIOR commented 4 years ago

Here you go:

config.txt

(I've changed the format to txt because github doesn't accept yaml)

awjuliani commented 4 years ago

Thanks for bringing this up. I was able to do some digging, and found that this is indeed a bug in our curriculum code. We'll be making a PR to fix it soon.

HPRIOR commented 4 years ago

Thanks for bringing this up. I was able to do some digging, and found that this is indeed a bug in our curriculum code. We'll be making a PR to fix it soon.

Any chance of a workaround or fix soon? I keep running into this bug when I try to train my agents with a curriculum

awjuliani commented 4 years ago

Hi @HPRIOR this has been fixed in the latest release of ML-Agents.

HPRIOR commented 4 years ago

@awjuliani Thanks

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.