Error from 'tensorflow_to_barracuda.py' file converting the model to an .nn one when using imitation learning and LSTM

[cristinadobre]

Hi! I'm trying to train an agent with behaviour cloning (imitation learning) with the use_recurrent (LSTM) feature active. For some reason it doesn't work, the error coming from tensorflow_to_barracuda.py file when it's trying to convert the model into an .nn one. I have replicated this using one of the imitation learning examples from the git repo (HallwayIL) by changing the 'use_recurrent' to be equal to true in the .yaml file. It works just fine, both in my project and in the examples if I have the use_recurrent set to false. I've tested on other example scenes that uses LSTM with ppo and that works fine too. Any idea why this is happening and how could I use LSTM with imitation learning?

More info below:

This is the error from the terminal:

Converting ./models/hallwayRecurrent-0/HallwayLearning/frozen_graph_def.pb to ./models/hallwayRecurrent-0/HallwayLearning.nn
Sorting model, may take a while........... Done!
IGNORED: Range unknown layer
Traceback (most recent call last):
  File "C:\Users\myname\AppData\Local\Continuum\anaconda3\envs\ml-agents\Scripts\mlagents-learn-script.py", line 11, in <module>
    load_entry_point('mlagents', 'console_scripts', 'mlagents-learn')()
  File "c:\users\myname\documents\ml-agents-master\ml-agents\mlagents\trainers\learn.py", line 417, in main
    run_training(0, run_seed, options, Queue())
  File "c:\users\myname\documents\ml-agents-master\ml-agents\mlagents\trainers\learn.py", line 255, in run_training
    tc.start_learning(env)
  File "c:\users\myname\documents\ml-agents-master\ml-agents\mlagents\trainers\trainer_controller.py", line 219, in start_learning
    self._export_graph()
  File "c:\users\myname\documents\ml-agents-master\ml-agents\mlagents\trainers\trainer_controller.py", line 129, in _export_graph
    self.trainers[brain_name].export_model()
  File "c:\users\myname\documents\ml-agents-master\ml-agents\mlagents\trainers\trainer.py", line 152, in export_model
    self.policy.export_model()
  File "c:\users\myname\documents\ml-agents-master\ml-agents\mlagents\trainers\tf_policy.py", line 230, in export_model
    tf2bc.convert(frozen_graph_def_path, self.model_path + ".nn")
  File "c:\users\myname\documents\ml-agents-master\ml-agents\mlagents\trainers\tensorflow_to_barracuda.py", line 1552, in convert
    i_model, args
  File "c:\users\myname\documents\ml-agents-master\ml-agents\mlagents\trainers\tensorflow_to_barracuda.py", line 1380, in process_model
    nodes, var_tensors, const_tensors, o_context
  File "c:\users\myname\documents\ml-agents-master\ml-agents\mlagents\trainers\tensorflow_to_barracuda.py", line 475, in <lambda>
    int(by_name(tensors, "/axis").data[0]), context.layer_ranks[inputs[0]]
IndexError: list index out of range

the .yaml file looks like this:

HallwayLearning: trainer: online_bc max_steps: 10000 summary_freq: 1000 brain_to_imitate: HallwayPlayer batch_size: 16 batches_per_epoch: 5 num_layers: 4 hidden_units: 64 use_recurrent: true sequence_length: 16 memory_size: 256

I run this on a windows machine with the latest version of ML agents ( from master branch cloned today). Python version is 3.6 and Unity version is 2019.1.3f1

The frozen_graph_def.pb file can be downloaded from here

Thank you!

[ervteng]

Hi @cristinadobre, we fixed this bug back in 0.9.1. Have you upgraded the Python package to the latest version?

[cristinadobre]

Hi @ervteng! I've cloned the repo and installed the packages using:

cd ml-agents-envs
pip install -e .
cd ..
cd ml-agents
pip install -e .

Is this what you mean?

[cristinadobre]

Hi @ervteng! I've cloned the repo and installed the packages using:

cd ml-agents-envs
pip install -e .
cd ..
cd ml-agents
pip install -e .

Is this what you mean?

I've also created another virtual environment and followed the steps from here . Unfortunately I still get that error.

[ervteng]

Yes, that looks fine! We've actually been able to reproduce your issue and are looking into it - it's been logged with internal tracking number MLA-93.

In the meantime, you can achieve something similar by using PPO+LSTM+GAIL. It will train slower but should achieve a better result. See: https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Training-Imitation-Learning.md

[cristinadobre]

Thank you for the quick reply @ervteng! I'll look into doing the PPO+LSTM+GAIL for now!

[ervteng]

This issue has been fixed in our develop branch - closing the issue for now. Thanks for reporting it!

P.S. We do consider BC a "legacy" feature that will be deprecated soon. PPO+GAIL and PreTraining can achieve the same results - and we suggest using that moving forward.

[cristinadobre]

Thanks for sorting it out! I'll check out the develop branch and consider using the proposed method.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.