arsenious
commented
6 years ago @mryellow
Open arsenious opened 6 years ago
arsenious
commented
6 years ago @mryellow
arsenious
commented
6 years ago Also, when I changed it to env = gym.make('MazeExplorerEat-v0')
Using TensorFlow backend. [2017-08-28 17:50:04,419] Making new env: MazeExplorerEat-v0 Traceback (most recent call last): File "/home/USERNAME/PycharmProjects/MazeExplorer/MazeTest.py", line 5, in env = gym.make('MazeExplorerEat-v0') File "/home/USERNAME/anaconda2/envs/tensorflow35/lib/python3.5/site-packages/gym/envs/registration.py", line 161, in make return registry.make(id) File "/home/USERNAME/anaconda2/envs/tensorflow35/lib/python3.5/site-packages/gym/envs/registration.py", line 119, in make env = spec.make() File "/home/USERNAME/anaconda2/envs/tensorflow35/lib/python3.5/site-packages/gym/envs/registration.py", line 85, in make cls = load(self._entry_point) File "/home/USERNAME/anaconda2/envs/tensorflow35/lib/python3.5/site-packages/gym/envs/registration.py", line 17, in load result = entry_point.load(False) File "/home/USERNAME/anaconda2/envs/tensorflow35/lib/python3.5/site-packages/pkg_resources/init.py", line 2316, in load return self.resolve() File "/home/USERNAME/anaconda2/envs/tensorflow35/lib/python3.5/site-packages/pkg_resources/init.py", line 2322, in resolve module = import(self.module_name, fromlist=['name'], level=0) File "/home/USERNAME/anaconda2/envs/tensorflow35/lib/python3.5/site-packages/gym_mazeexplorer/envs/init.py", line 1, in from gym_mazeexplorer.envs.mazeexplorer_env import MazeExplorerEnv File "/home/USERNAME/anaconda2/envs/tensorflow35/lib/python3.5/site-packages/gym_mazeexplorer/envs/mazeexplorer_env.py", line 8, in import mazeexp as mx File "/home/USERNAME/anaconda2/envs/tensorflow35/lib/python3.5/site-packages/mazeexp/init.py", line 1, in from mazeexp.engine.mazeexp import MazeExplorer File "/home/USERNAME/anaconda2/envs/tensorflow35/lib/python3.5/site-packages/mazeexp/engine/mazeexp.py", line 10, in import config ImportError: No module named 'config'
arsenious
commented
6 years ago I'm using Python 3.5 so I think it was an import issue Changing the line import config -> import mazeexp.engine.config worked
But this new error comes and I can't file assets folder
Using TensorFlow backend.
[2017-08-28 18:00:26,573] Making new env: MazeExplorerEat-v0
Traceback (most recent call last):
File "/home/USERNAME/PycharmProjects/MazeExplorer/MazeTest.py", line 5, in
mryellow
commented
6 years ago I'm using Python 3.5
That's possibly an issue, believe I had to stick with 2.7 due to dependencies somewhere along the line. Give it a go though.
"player": pyglet.image.load(os.path.join(script_dir, 'assets', 'player7.png')),
Okay so that exists in https://github.com/mryellow/maze_explorer/tree/master/mazeexp/engine/assets ... Have seen this error before, guess it's a dev/production environment issue.
That part should be including the assets files, confirmed they show in the distribution zip file...
I'll see if I can duplicate it and fix the build config.
A work-around which should work would be to clone the gym and environment repo, then pip install -e .. That will link direct to the source and the file will be in the expected path.
I was able to get it working with adding import gym_mazeexplorer to examples/openai_gym.py:
python ~/src/tensorforce_fork/examples/openai_gym.py -m logs/agent001 -s logs/agents/agent001 --monitor-video 100 MazeExplorerEat-v0 -a DQNAgent -c logs/agent.json -n logs/network_flat_lstm.json
agent.json
{
"preprocessing": null,
"exploration": {
"type": "epsilon_anneal",
"epsilon": 1.0,
"epsilon_final": 0.01,
"epsilon_timesteps": 2e6,
"start_after": 5000
},
"batch_size": 64,
"memory_capacity": 1000000,
"memory": "prioritized_replay",
"update_frequency": 4,
"first_update": 5000,
"repeat_update": 1,
"target_update_frequency": 10000,
"discount": 0.95,
"learning_rate": 0.0025,
"optimizer": {
"type": "rmsprop",
"momentum": 0.95,
"epsilon": 0.01
},
"tf_saver": true,
"tf_summary": null,
"log_level": "info",
"update_target_weight": 1.0,
"double_dqn": false,
"clip_loss": 0.0
}network_flat_lstm.json
[
{
"type": "flatten"
},
{
"type": "dense",
"size": 64
},
{
"type": "dense",
"size": 64
},
{
"type": "lstm"
}
]Although classic DQN has trouble with the reward function phrased how it is currently (where only apples/poison/walls have rewards but no constant proximity signal like Karpathy used).
Not certain the memory config is making it into agent with current build, complains about Configuration values not accessed: first_update, update_frequency, repeat_update, memory, batch_size, memory_capacity, tf_saver.
mryellow
commented
6 years ago Deployed a new version 0.0.10 which includes the asset files in both the source and binary distributions.
https://github.com/mryellow/maze_explorer/blob/ab8a25ccd05105d2fe57e0213d690cfc07e45827/setup.py#L13
arsenious
commented
6 years ago Thank you for the help! I'll check it out and let you know. I'm working on Deep RL for maze solving and I think this project has great potential as a benchmark environment
mryellow
commented
6 years ago great potential as a benchmark environment
I see the apples/poison thing as being very similar to DeepMind's Labyrinth but with a much smaller input state space, making it good for smaller/faster experiments.
Think really need to reign in the default reward function for benchmark use, something easily learnable which provides meaningful scores or reward graph.
Currently running with -10 on wall collisions, without setting a terminal state (so it keeps running into walls). Which results in a reward graph starting at around -1000 for 500 timesteps and approaching 0 as epsilon exploration gets out of the way. Something which starts at 0 and continues to improve all the way to super-human is probably needed to make it a good benchmark.
Has been handy running it this way for discovering bugs in velocity code or wall collisions, but think need to cut all those "running into a wall" experiences by setting a terminal flag and having more prioritised experiences relating to finding apples rather than avoiding walls.
I'll do a few more experiments between these wall rewards and Karpathy's wall proximity based reward. Aiming for something which can be effectively trained on a classic DQN without A3C (his proximity reward results in smooth forward reward seeking behaviour after 1m experiences, where wall reward can result in a lot of spinning in circles).
Then when it comes to reward functions which are a little more rarely rewarding (like second exploration/escape game mode) there is some A3C testing needed. However I had TF device errors on first attempt running that with TensorForce, not sure what needs changing in the config.
Hi
I would really like to use this environment for Deep RL reserach purposes. But I'm not able to get it to work. Please help. Thanks
Using TensorFlow backend. [2017-08-28 17:41:07,956] Making new env: MazeExplorer-v0 Traceback (most recent call last): File "/home/USERNAME/anaconda2/envs/tensorflow35/lib/python3.5/site-packages/gym/envs/registration.py", line 137, in spec return self.env_specs[id] KeyError: 'MazeExplorer-v0'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/home/USERNAME/PycharmProjects/MazeExplorer/MazeTest.py", line 5, in
env = gym.make('MazeExplorer-v0')
File "/home/USERNAME/anaconda2/envs/tensorflow35/lib/python3.5/site-packages/gym/envs/registration.py", line 161, in make
return registry.make(id)
File "/home/USERNAME/anaconda2/envs/tensorflow35/lib/python3.5/site-packages/gym/envs/registration.py", line 118, in make
spec = self.spec(id)
File "/home/USERNAME/anaconda2/envs/tensorflow35/lib/python3.5/site-packages/gym/envs/registration.py", line 147, in spec
raise error.UnregisteredEnv('No registered env with id: {}'.format(id))
gym.error.UnregisteredEnv: No registered env with id: MazeExplorer-v0
Process finished with exit code 1