Crash when using curiosity

[bitzoic]

Hello, I am attempting to train my agent using curiosity and have added curiosity to the reward signals in the trainer_config. After adding the curiosity section I now am unable to being training. If I remove it, it works perfectly fine again.

I am using Python 3.6, ML-Agents 0.14.1, and Unity 2019.2.8f1.

This is the error I am getting in the command prompt after entering "mlagents-learn config/trainer_config.yaml --run-id=eigthRun --train --time-scale 1" to train:

Traceback (most recent call last):
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\ThePr\AppData\Local\Programs\Python\Python36\Scripts\mlagents-learn.exe\__main__.py", line 7, in <module>
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\mlagents\trainers\learn.py", line 479, in main
    run_cli(parse_command_line())
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\mlagents\trainers\learn.py", line 240, in parse_command_line
    return RunOptions.from_argparse(args)
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\mlagents\trainers\learn.py", line 213, in from_argparse
    argparse_args["trainer_config"] = load_config(trainer_config_path)
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\mlagents\trainers\trainer_util.py", line 172, in load_config
    return _load_config(data_file)
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\mlagents\trainers\trainer_util.py", line 188, in _load_config
    return yaml.safe_load(fp)
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\yaml\__init__.py", line 162, in safe_load
    return load(stream, SafeLoader)
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\yaml\__init__.py", line 114, in load
    return loader.get_single_data()
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\yaml\constructor.py", line 41, in get_single_data
    node = self.get_single_node()
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\yaml\composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\yaml\composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\yaml\composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\yaml\composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\yaml\composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\yaml\composer.py", line 127, in compose_mapping_node
    while not self.check_event(MappingEndEvent):
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\yaml\parser.py", line 98, in check_event
    self.current_event = self.state()
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\yaml\parser.py", line 428, in parse_block_mapping_key
    if self.check_token(KeyToken):
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\yaml\scanner.py", line 116, in check_token
    self.fetch_more_tokens()
  File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\yaml\scanner.py", line 260, in fetch_more_tokens
    self.get_mark())
yaml.scanner.ScannerError: while scanning for the next token
found character '\t' that cannot start any token
  in "config/trainer_config.yaml", line 20, column 1

This is what my trainer_config looks like:

trainer: ppo batch_size: 256 beta: 5.0e-3 buffer_size: 4096 epsilon: 0.2 hidden_units: 256 lambd: 0.95 learning_rate: 5.0e-4 learning_rate_schedule: constant max_steps: 1e7 memory_size: 256 normalize: false num_epoch: 3 num_layers: 2 time_horizon: 64 sequence_length: 64 summary_freq: 10000 use_recurrent: false vis_encode_type: simple reward_signals: extrinsic: strength: 1.0 gamma: 0.99 curiosity: strength: 0.02 gamma: 0.99 encoding_size:

I am unsure of where to go from here. Thank you!

[vincentpierre]

Hi.

Please use the template for issues that is provided when creating a github issue. If none of the templates fit your issue, you should post on the Unity Forums. The error you are getting seems to come from parsing the yaml config file as suggested by

File "c:\users\thepr\appdata\local\programs\python\python36\lib\site-packages\yaml\__init__.py", line 162, in safe_load
    return load(stream, SafeLoader)

Yaml is a rather capricious format, you need to be careful about indent for example. Make sure your yaml file is formatted exactly like the ones provided in our example environments :

Pyramids:
    summary_freq: 30000
    time_horizon: 128
    batch_size: 128
    buffer_size: 2048
    hidden_units: 512
    num_layers: 2
    beta: 1.0e-2
    max_steps: 1.0e7
    num_epoch: 3
    reward_signals:
        extrinsic:
            strength: 1.0
            gamma: 0.99
        curiosity:
            strength: 0.02
            gamma: 0.99
            encoding_size: 256

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.