'NoneType' object has no attribute 'get_qbaseline_sim

[malagori]

Hi, I am trying to run the example qprop_cartpole.py but it breaks with an error (line 122, npo.py) below. In the example script, baseline is already set with the LinearFeatureBaseLine. I am not sure whats going wrong here.

Can any one help?

Thanks in advance!

/sandbox/rocky/tf/launchers$ python qprop_cartpole.py python /home/meh/Documents/RL-work/code/rllab/rllabplusplus/scripts/run_experiment_lite.py --exp_name 'experiment_2017_03_06_15_26_08_0001' --args_data 'gANjcmxsYWIubWlzYy5pbnN0cnVtZW50ClN0dWJNZXRob2RDYWxsCnEAKYFxAX1xAihYCAAAAF9fa3dhcmdzcQN9cQRYBgAAAF9fYXJnc3EFKGNybGxhYi5taXNjLmluc3RydW1lbnQKU3R1Yk9iamVjdApxBimBcQd9cQgoWAsAAABwcm94eV9jbGFzc3EJY3NhbmRib3gucm9ja3kudGYuYWxnb3MudHJwbwpUUlBPCnEKWAYAAABrd2FyZ3NxC31xDChYAgAAAHFmcQ1oBimBcQ59cQ8oaAljc2FuZGJveC5yb2NreS50Zi5xX2Z1bmN0aW9ucy5jb250aW51b3VzX21scF9xX2Z1bmN0aW9uCkNvbnRpbnVvdXNNTFBRRnVuY3Rpb24KcRBoC31xEVgIAAAAZW52X3NwZWNxEmNybGxhYi5taXNjLmluc3RydW1lbnQKU3R1YkF0dHIKcRMpgXEUfXEVKFgKAAAAX2F0dHJfbmFtZXEWWAQAAABzcGVjcRdYBAAAAF9vYmpxGGgGKYFxGX1xGihoCWNzYW5kYm94LnJvY2t5LnRmLmVudnMuYmFzZQpUZkVudgpxG2gLfXEcWAsAAAB3cmFwcGVkX2VudnEdaAYpgXEefXEfKGgJY3JsbGFiLmVudnMubm9ybWFsaXplZF9lbnYKTm9ybWFsaXplZEVudgpxIGgLfXEhWAMAAABlbnZxImgGKYFxI31xJChoCWNybGxhYi5lbnZzLmJveDJkLmNhcnRwb2xlX2VudgpDYXJ0cG9sZUVudgpxJWgLfXEmWAQAAABhcmdzcScpdWJzaCcpdWJzaCcpdWJ1YnNoJyl1YlgIAAAAZGlzY291bnRxKEc/764UeuFHrmgiaBlYBgAAAHBvbGljeXEpaAYpgXEqfXErKGgJY3NhbmRib3gucm9ja3kudGYucG9saWNpZXMuZ2F1c3NpYW5fbWxwX3BvbGljeQpHYXVzc2lhbk1MUFBvbGljeQpxLGgLfXEtKFgMAAAAaGlkZGVuX3NpemVzcS5LIEsghnEvaBJoEymBcTB9cTEoaBZoF2gYaBl1YlgEAAAAbmFtZXEyaCl1aCcpdWJYCQAAAHN0ZXBfc2l6ZXEzRz+EeuFHrhR7WA8AAABtYXhfcGF0aF9sZW5ndGhxNEtkWAoAAABiYXRjaF9zaXplcTVNoA9YBQAAAG5faXRycTZLKFgIAAAAYmFzZWxpbmVxN2gGKYFxOH1xOShoCWNybGxhYi5iYXNlbGluZXMubGluZWFyX2ZlYXR1cmVfYmFzZWxpbmUKTGluZWFyRmVhdHVyZUJhc2VsaW5lCnE6aAt9cTtoEmgTKYFxPH1xPShoFmgXaBhoGXVic2gnKXVidWgnKXViWAUAAAB0cmFpbnE+KX1xP3RxQHViLg==' --log_dir '/home/meh/Documents/RL-work/code/rllab/rllabplusplus/data/local/experiment/experiment_2017_03_06_15_26_08_0001--s-1' --seed '1' --use_cloudpickle 'False' --n_parallel '4' /home/meh/anaconda2/envs/rllab3/lib/python3.5/site-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module. "downsample module has been moved to the theano.tensor.signal.pool module.") using seed 1 2017-03-06 15:26:10.228756 CET | Setting seed to 1 using seed 1 2017-03-06 15:26:10.228982 CET | Setting seed to 2 using seed 2 2017-03-06 15:26:10.230003 CET | Setting seed to 4 2017-03-06 15:26:10.229928 CET | Setting seed to 3 using seed 4 using seed 3 Traceback (most recent call last): File "/home/meh/Documents/RL-work/code/rllab/rllabplusplus/scripts/run_experiment_lite.py", line 137, in run_experiment(sys.argv) File "/home/meh/Documents/RL-work/code/rllab/rllabplusplus/scripts/run_experiment_lite.py", line 124, in run_experiment maybe_iter = concretize(data) File "/home/meh/Documents/RL-work/code/rllab/rllabplusplus/rllab/misc/instrument.py", line 1217, in concretize return method(*args, **kwargs) File "/home/meh/Documents/RL-work/code/rllab/rllabplusplus/sandbox/rocky/tf/algos/batch_polopt.py", line 180, in train self.init_opt() File "/home/meh/Documents/RL-work/code/rllab/rllabplusplus/sandbox/rocky/tf/algos/npo.py", line 122, in init_opt qbaseline_info = self.qf_baseline.get_qbaseline_sim( AttributeError: 'NoneType' object has no attribute 'get_qbaseline_sim'

[rlbayes]

For Q-Prop, you need to pass in another argument, qf_baseline=. For details, please check https://github.com/shaneshixiang/rllabplusplus/blob/master/sandbox/rocky/tf/launchers/launcher_stub_utils.py#L121. I have some refactored codes while adding new features (yet to push), where I removed this weird code structure.

[iDurugkar]

Hi, similar problem. Can you tell me where I can pass in this argument qf_baseline? Do I do it when I create the TRPO() object at https://github.com/shaneshixiang/rllabplusplus/blob/master/sandbox/rocky/tf/launchers/qprop_cartpole.py#L25?

Or someplace else?

[rlbayes]

I have committed a change. Please let me know if problem persists. https://github.com/shaneshixiang/rllabplusplus/commit/8fb07316f6d846696b8edf3620b13493793d5517

[iDurugkar]

Yes its working now. Thanks!