Could we specify the ternsorflow and gym versions? Thanks!

[wangpd]

I found the codes could not work with late gym versions, e.g., gym 0.6.0 could not work, while 0.5.6 could work.

In terms of tensorflow versions, I found the codes could not work with 1.1.0. After changing some codes, the codes could work with 1.1.0 now.

I am wondering what are the tensorflow and gym versions during the development of the codes. I have already made the codes work with tensorflow 1.1.0 and gym 0.9.2. If needed, I could create a pull request to merge my changes to the repo.

Thank you!

[spiglerg]

Could you please post the full error you get? I have recently tried the code (albeit a local version, not the git one, so it may be slightly different) on the new gym and tensorflow and I think it was working fine.

The snapshot was created with the older versions, though, so that may easily fail now.

[wangpd]

Hi @spiglerg,

Thanks for the reply! And also thanks a lot for this repo!

Sure, I am posting some errors I had met here and my environment settings, though they are not all. Because after fixing one error, I will see the next one. Of course, finally I have fixed all of them to work with tensorflow 1.1.0 and gym 0.9.2 on AWS GPU instances.

My environment settings: Server: AWS p2.xlarge GPU instance Image: Bitfusion Ubuntu 14 TensorFlow: https://aws.amazon.com/marketplace/pp/B01EYKBEQ0?ref=cns_srchrow Tensorflow version: 1.1.0 Openai gym version: 0.9.2

The first error I got (a tensorflow error):

$ git clone https://github.com/spiglerg/DQN_DDQN_Dueling_and_DDPG_Tensorflow.git
Cloning into 'DQN_DDQN_Dueling_and_DDPG_Tensorflow'...
remote: Counting objects: 77, done.
remote: Total 77 (delta 0), reused 0 (delta 0), pack-reused 77
Unpacking objects: 100% (77/77), done.
Checking connectivity... done.
$ cd DQN_DDQN_Dueling_and_DDPG_Tensorflow
$ git branch
* master
$ git log -1
commit 346aa144778dc296806292a36a53be29803ddae9
Author: Giacomo Spigler <spiglerg@gmail.com>
Date:   Tue Feb 14 13:28:09 2017 +0000

    removed deprecated argument
$ python gym_dqn_atari.py

Usage:
   gym_dqn_atari.py  [optional: path_to_ckpt_file] [optional: True/False test mode]

Traceback (most recent call last):
  File "gym_dqn_atari.py", line 53, in <module>
    journalist = tf.train.SummaryWriter(LOG_DIR)
AttributeError: 'module' object has no attribute 'SummaryWriter'

$ sudo pip show gym
Name: gym
Version: 0.9.2
Summary: The OpenAI Gym: A toolkit for developing and comparing your reinforcement learning agents.
Home-page: https://github.com/openai/gym
Author: OpenAI
Author-email: gym@openai.com
License: UNKNOWN
Location: /usr/local/lib/python2.7/dist-packages
Requires: requests, six, numpy, pyglet
$ sudo pip show tensorflow
Name: tensorflow
Version: 1.1.0
Summary: TensorFlow helps the tensors flow
Home-page: http://tensorflow.org/
Author: Google Inc.
Author-email: opensource@google.com
License: Apache 2.0
Location: /usr/local/lib/python2.7/dist-packages
Requires: six, protobuf, werkzeug, mock, numpy, wheel

This error is due to the tensorflow version. Tensorflow 1.1.0 uses:

journalist = tf.summary.FileWriter(LOG_DIR)

instead of

journalist = tf.train.SummaryWriter(LOG_DIR)

which is on the master branch now.

After fixing this error, I also met other errors from tensorflow and openai gym. I have already adapted the codes on the master branch locally to work with tensorflow 1.1.0 and gym 0.9.2.

Btw, another issue that I have spent quite a while to fix is that when we do the tests we need to reset the gym env. However, gym would throw errors like (this error may not exist with some old version of the gym module):

frames:  100000     games:  140     speed:  53.6304661459  frames/second
--
epoch:  frames= 100000    games= 140
Traceback (most recent call last):
File "gym_dqn_atari.py", line 183, in <module>
state = env.reset()
File "/home/ubuntu/projects/DQN_DDQN_Dueling_and_DDPG_Tensorflow/modules/env_utils.py", line 52, in reset
return self.process_frame(self.env.reset())
File "/usr/local/lib/python2.7/dist-packages/gym/core.py", line 139, in reset
self.monitor._before_reset()
File "/usr/local/lib/python2.7/dist-packages/gym/monitoring/monitor.py", line 271, in _before_reset
self.stats_recorder.before_reset()
File "/usr/local/lib/python2.7/dist-packages/gym/monitoring/stats_recorder.py", line 65, in before_reset
raise error.Error("Tried to reset environment which is not done. While the monitor is active for {}, you cannot call reset() unless the episode is over.".format(self.env_id))
gym.error.Error: Tried to reset environment which is not done. While the monitor is active for Seaquest-v0, you cannot call reset() unless the episode is over.

I have fixed this env reset issue following: https://github.com/rll/rllab/issues/87#issuecomment-282519288 https://github.com/rll/rllab/blob/master/rllab/envs/gym_env.py

Thank you!