search

spragunr / deep_q_rl

Theano-based implementation of Deep Q-learning
BSD 3-Clause "New" or "Revised" License
1.08k stars 348 forks source link

theano.gradient.consider_constant is deprecated #14

Closed ndcomplete closed 9 years ago

ndcomplete commented 9 years ago

According to this, theano.gradient.consider_constant is deprecated. zero_grad() or disconnected_grad() should be used instead.

coventry commented 9 years ago

I think I just ran into this issue. Error message is below. It'll probably be effectively a silent failure for some people, as the training epoch: x steps_left: y output continues and instantly pushes the traceback off the screen.

(theano)lap% python -i ale_run.py --exp_pref data -r `pwd`/breakout.bin --display-screen
RL-Glue Version 3.04, Build 909
RL-Glue is listening for connections on port=4096
A.L.E: Arcade Learning Environment (version 0.4)
[Powered by Stella]
Use -help for help screen.
Warning: couldn't load settings file: ./stellarc
Game console created:
  ROM file:  /home/coventry/ml/games/deep_q_rl/deep_q_rl/breakout.bin
  Cart Name: Breakout - Breakaway IV (1978) (Atari)
  Cart MD5:  f34f08e5eb96e500e851a80be3277a56
  Display Format:  AUTO-DETECT ==> NTSC
  ROM Size:        2048
  Bankswitch Type: AUTO-DETECT ==> 2K

RL-Glue Python Experiment Codec Version: 2.02 (Build 738)
    Connecting to 127.0.0.1 on port 4096...
    RL-Glue :: Experiment connected.
Screen Display Active. Press 'h' for help.
Running ROM file...
Random Seed: Time
Game will be controlled through RL-Glue.
Initializing ALE RL-Glue ...
    RL-Glue :: Environment connected.
Using gpu device 0: GeForce GTX 650
RL-Glue Python Agent Codec Version: 2.02 (Build 738)
    Connecting to 127.0.0.1 on port 4096...
     Agent Codec Connected
    RL-Glue :: Agent connected.
(32, 4, 84, 84)
(4, 84, 84, 32)
(16, 20.0, 20.0, 32)
(32, 9.0, 9.0, 32)
(32, 32, 9.0, 9.0)
(32, 256)
(32, 18)
Traceback (most recent call last):
  File "./rl_glue_ale_agent.py", line 456, in <module>
    main()
  File "./rl_glue_ale_agent.py", line 452, in main
    AgentLoader.loadAgent(NeuralAgent())
  File "/home/coventry/Env/theano/local/lib/python2.7/site-packages/rlglue/agent/AgentLoader.py", line 58, in loadAgent
    client.runAgentEventLoop()
  File "/home/coventry/Env/theano/local/lib/python2.7/site-packages/rlglue/agent/ClientAgent.py", line 144, in runAgentEventLoop
    switch[agentState](self)
  File "/home/coventry/Env/theano/local/lib/python2.7/site-packages/rlglue/agent/ClientAgent.py", line 137, in <lambda>
    Network.kAgentInit: lambda self: self.onAgentInit(),
  File "/home/coventry/Env/theano/local/lib/python2.7/site-packages/rlglue/agent/ClientAgent.py", line 43, in onAgentInit
    self.agent.agent_init(taskSpec)
  File "./rl_glue_ale_agent.py", line 169, in agent_init
    self.network = self._init_network()
  File "./rl_glue_ale_agent.py", line 205, in _init_network
    approximator='cuda_conv')
  File "/play/coventry/ml/games/deep_q_rl/deep_q_rl/cnn_q_learner.py", line 170, in __init__
    target = theano.gradient.consider_constant(target)
AttributeError: 'module' object has no attribute 'consider_constant'
training epoch:  1 steps_left:  50000
spragunr commented 9 years ago

Are you using the github version of Theano?

coventry commented 9 years ago

No, and pip insists on giving me version 0.6.0 for some reason, even though yolk sees 0.7.0rc2. I will try building from source.

spragunr commented 9 years ago

This issue is addressed by PR #21.