ImportError: No module named gym_pathfinding

[fanyuzeng]

where is the gym_pathfinding module? I can not find it in github. Thank you so much.

[didibear]

Hi, it is here https://github.com/DidiBear/gym-pathfinding

[fanyuzeng]

ok, thank you very much.

[didibear]

Tell me if you have questions, because my implementation doesn't work well compare to the results in the paper It's surely due to the batch training which is not done perfectly

[fanyuzeng]

ok, thank you so much.

[fanyuzeng]

Could you write a detailed readme? In addition, I have a error as following: IOError: [Errno 2] No such file or directory: './data/dataset.pkl' There is on this directory in you projection.

[didibear]

The dataset.pkl is generated by the generate_dataset.py. But yeah I will write a more detailed readme :)

[fanyuzeng]

Ok, thank you very much.

[fanyuzeng]

I install gym-pathfinding successfully, and the terminal shows following: cr@cr-server:~/Downloads/gym-pathfinding-master$ sudo pip install -e . The directory '/home/cr/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. The directory '/home/cr/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. Looking in indexes: https://mirrors.ustc.edu.cn/pypi/web/simple Obtaining file:///home/cr/Downloads/gym-pathfinding-master Requirement already satisfied: gym==0.9.7 in /usr/local/lib/python2.7/dist-packages (from gym-pathfinding==0.0.1) (0.9.7) Requirement already satisfied: cython in /usr/lib/python2.7/dist-packages (from gym-pathfinding==0.0.1) (0.20.1.post0) Requirement already satisfied: numpy in /usr/local/lib/python2.7/dist-packages (from gym-pathfinding==0.0.1) (1.14.5) Requirement already satisfied: scipy in /usr/local/lib/python2.7/dist-packages/scipy-0.19.1-py2.7-linux-x86_64.egg (from gym-pathfinding==0.0.1) (0.19.1) Requirement already satisfied: pygame in /usr/local/lib/python2.7/dist-packages (from gym-pathfinding==0.0.1) (1.9.4) Requirement already satisfied: requests>=2.0 in /usr/lib/python2.7/dist-packages (from gym==0.9.7->gym-pathfinding==0.0.1) (2.2.1) Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from gym==0.9.7->gym-pathfinding==0.0.1) (1.11.0) Requirement already satisfied: pyglet>=1.2.0 in /usr/local/lib/python2.7/dist-packages (from gym==0.9.7->gym-pathfinding==0.0.1) (1.2.4) Installing collected packages: gym-pathfinding Found existing installation: gym-pathfinding 0.0.1 Uninstalling gym-pathfinding-0.0.1: Successfully uninstalled gym-pathfinding-0.0.1 Running setup.py develop for gym-pathfinding Successfully installed gym-pathfinding

However, I can't import gym-pathfinding successfully. How to resolve this problem? Following shows the error: cr@cr-server:~/Downloads/gym-pathfinding-master$ python Python 2.7.6 (default, Nov 23 2017, 15:49:48) [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" for more information.

import gym-pathfinding File "", line 1 import gym-pathfinding ^ SyntaxError: invalid syntax

[didibear]

Hi, you setup the library well, that's just the importing that should be :

import gym_pathfinding # instead of gym-pathfinding

That's a common missunderstanding between - in files and _ in code. Run the python example.py just to see if everything is ok

[fanyuzeng]

It's ok, thank you : )

[fanyuzeng]

I run your code on TITANX, and the result is following which is close to that of the original paper. Start training... Epoch: 1 (993.1 s): Train Loss: 48.70277 Train accuracy: 43.29% Epoch: 2 (988.2 s): Train Loss: 30.65247 Train accuracy: 71.29% Epoch: 3 (987.5 s): Train Loss: 20.32545 Train accuracy: 81.21% Epoch: 4 (986.9 s): Train Loss: 15.16959 Train accuracy: 85.58% Epoch: 5 (985.1 s): Train Loss: 12.79127 Train accuracy: 87.56% Epoch: 6 (991.8 s): Train Loss: 11.25151 Train accuracy: 88.90% Epoch: 7 (988.6 s): Train Loss: 10.36887 Train accuracy: 89.53% Epoch: 8 (988.2 s): Train Loss: 9.57739 Train accuracy: 90.16% Epoch: 9 (990.0 s): Train Loss: 9.02159 Train accuracy: 90.54% Epoch: 10 (988.6 s): Train Loss: 8.43890 Train accuracy: 91.15% Epoch: 11 (986.8 s): Train Loss: 7.99584 Train accuracy: 91.53% Epoch: 12 (989.3 s): Train Loss: 7.85253 Train accuracy: 91.68% Epoch: 13 (987.2 s): Train Loss: 7.41661 Train accuracy: 92.08% Epoch: 14 (986.6 s): Train Loss: 7.04018 Train accuracy: 92.55% Epoch: 15 (983.0 s): Train Loss: 6.91027 Train accuracy: 92.65% Epoch: 16 (988.2 s): Train Loss: 6.67543 Train accuracy: 92.94% Epoch: 17 (985.2 s): Train Loss: 6.40542 Train accuracy: 93.16% Epoch: 18 (986.3 s): Train Loss: 6.29157 Train accuracy: 93.40% Epoch: 19 (983.0 s): Train Loss: 6.16197 Train accuracy: 93.55% Epoch: 20 (987.2 s): Train Loss: 5.90023 Train accuracy: 93.83% Epoch: 21 (991.8 s): Train Loss: 5.83335 Train accuracy: 93.96% Epoch: 22 (992.7 s): Train Loss: 5.67282 Train accuracy: 94.16% Epoch: 23 (987.4 s): Train Loss: 5.45612 Train accuracy: 94.40% Epoch: 24 (992.9 s): Train Loss: 5.29960 Train accuracy: 94.61% Epoch: 25 (989.7 s): Train Loss: 5.10882 Train accuracy: 94.84% Epoch: 26 (990.1 s): Train Loss: 5.12046 Train accuracy: 94.88% Epoch: 27 (987.0 s): Train Loss: 5.03132 Train accuracy: 95.02% Epoch: 28 (987.4 s): Train Loss: 4.91650 Train accuracy: 95.13% Epoch: 29 (987.8 s): Train Loss: 4.85280 Train accuracy: 95.31% Epoch: 30 (987.3 s): Train Loss: 4.73644 Train accuracy: 95.39% Epoch: 31 (985.6 s): Train Loss: 4.70844 Train accuracy: 95.42% Epoch: 32 (989.0 s): Train Loss: 4.51320 Train accuracy: 95.65%

I go on training it.

[didibear]

Hi,

Here you show the accuracy during training, but the relevent part is the accuracy during testing, which was around 60% for me (vs >90% in paper) Here the results is either an overfitting of the training set (what we don't want) or a generalization (what we want), or both. For me, my agent was able to avoid walls, and reach goal when it see it, but the exploration of the map was hazardous.

The problem surely came from my batch training, which batch episodes with the same size. So the last frame of an episode, where the agent reached the goal, is repeated until the the batch width is reached, causing unrelevent data. For example, if episode size = 10, 1 mean the agent search the goal, and 0 mean the agent attain the goal, we have data like : 1111100000 1111111100 1110000000 1111110000 ... All zeros are unrelevent.

I tried to implement a way to ignore those data using bit masks for each episode but it doesn't change results. Maybe I miss something, such as the DNC memory, in which I did not study what happen.

And I do not think single episode training is a good way to go, as it make train accuracy less meaningful.

Hope this will help

[fanyuzeng]

Ok, thank you so much for you detailed answer. I think there are some tricks to train the agent, which was not shown in the paper. I will analyze this paper and study your code. I have studied DNC before, but I have not used it successfully. How do you think about DNC? Do you think it is a good type of external memory, compared to memory network?

[didibear]

I find the idea of external memory really promising and necessary in for advanced reinforcement learning, notably through some works using it. However, the hard part is not to store information, but using it for long term planning. Sadly, as I'm a newcomer, I never experienced the performance of such model, neither in DNC nor others, I cannot compare it. The intention behind my implementation mostly was to learn DRL.

I do not know your background, but here are really good references and works that you'll surely find interesting :

• Good explanation about DNC : https://greydanus.github.io/2017/02/27/differentiable-memory-and-the-brain/
• Good explanation about attention mechanism (Neural Turing Machine part) : https://distill.pub/2016/augmented-rnns/
• Neural Map (02/2017) : conference / paper
• Neural SLAM (06/2017) : quick presentation / paper

[fanyuzeng]

Thank you so much. The explanation about DNC and attention mechanism is very good. I have read Neural Map and Neural SLAM, which used external memory successfully. I will read the two paper again, as I want to do research on external memory to improve the transfer ability of agents. Following is the result of your code on my TITIANX: Start training... Epoch: 1 (993.1 s): Train Loss: 48.70277 Train accuracy: 43.29% Epoch: 2 (988.2 s): Train Loss: 30.65247 Train accuracy: 71.29% Epoch: 3 (987.5 s): Train Loss: 20.32545 Train accuracy: 81.21% Epoch: 4 (986.9 s): Train Loss: 15.16959 Train accuracy: 85.58% Epoch: 5 (985.1 s): Train Loss: 12.79127 Train accuracy: 87.56% Epoch: 6 (991.8 s): Train Loss: 11.25151 Train accuracy: 88.90% Epoch: 7 (988.6 s): Train Loss: 10.36887 Train accuracy: 89.53% Epoch: 8 (988.2 s): Train Loss: 9.57739 Train accuracy: 90.16% Epoch: 9 (990.0 s): Train Loss: 9.02159 Train accuracy: 90.54% Epoch: 10 (988.6 s): Train Loss: 8.43890 Train accuracy: 91.15% Epoch: 11 (986.8 s): Train Loss: 7.99584 Train accuracy: 91.53% Epoch: 12 (989.3 s): Train Loss: 7.85253 Train accuracy: 91.68% Epoch: 13 (987.2 s): Train Loss: 7.41661 Train accuracy: 92.08% Epoch: 14 (986.6 s): Train Loss: 7.04018 Train accuracy: 92.55% Epoch: 15 (983.0 s): Train Loss: 6.91027 Train accuracy: 92.65% Epoch: 16 (988.2 s): Train Loss: 6.67543 Train accuracy: 92.94% Epoch: 17 (985.2 s): Train Loss: 6.40542 Train accuracy: 93.16% Epoch: 18 (986.3 s): Train Loss: 6.29157 Train accuracy: 93.40% Epoch: 19 (983.0 s): Train Loss: 6.16197 Train accuracy: 93.55% Epoch: 20 (987.2 s): Train Loss: 5.90023 Train accuracy: 93.83% Epoch: 21 (991.8 s): Train Loss: 5.83335 Train accuracy: 93.96% Epoch: 22 (992.7 s): Train Loss: 5.67282 Train accuracy: 94.16% Epoch: 23 (987.4 s): Train Loss: 5.45612 Train accuracy: 94.40% Epoch: 24 (992.9 s): Train Loss: 5.29960 Train accuracy: 94.61% Epoch: 25 (989.7 s): Train Loss: 5.10882 Train accuracy: 94.84% Epoch: 26 (990.1 s): Train Loss: 5.12046 Train accuracy: 94.88% Epoch: 27 (987.0 s): Train Loss: 5.03132 Train accuracy: 95.02% Epoch: 28 (987.4 s): Train Loss: 4.91650 Train accuracy: 95.13% Epoch: 29 (987.8 s): Train Loss: 4.85280 Train accuracy: 95.31% Epoch: 30 (987.3 s): Train Loss: 4.73644 Train accuracy: 95.39% Epoch: 31 (985.6 s): Train Loss: 4.70844 Train accuracy: 95.42% Epoch: 32 (989.0 s): Train Loss: 4.51320 Train accuracy: 95.65% Epoch: 33 (988.3 s): Train Loss: 4.47834 Train accuracy: 95.68% Epoch: 34 (983.3 s): Train Loss: 4.43606 Train accuracy: 95.69% Epoch: 35 (986.4 s): Train Loss: 4.28522 Train accuracy: 95.90% Epoch: 36 (987.4 s): Train Loss: 4.27190 Train accuracy: 95.93% Epoch: 37 (987.6 s): Train Loss: 4.27363 Train accuracy: 95.92% Epoch: 38 (987.3 s): Train Loss: 4.14151 Train accuracy: 96.06% Epoch: 39 (984.1 s): Train Loss: 4.09524 Train accuracy: 96.08% Epoch: 40 (988.0 s): Train Loss: 3.99667 Train accuracy: 96.22% Epoch: 41 (986.8 s): Train Loss: 3.92269 Train accuracy: 96.26% Epoch: 42 (984.6 s): Train Loss: 3.89696 Train accuracy: 96.33% Epoch: 43 (987.3 s): Train Loss: 3.81214 Train accuracy: 96.42% Epoch: 44 (984.8 s): Train Loss: 3.78048 Train accuracy: 96.42% Epoch: 45 (983.9 s): Train Loss: 3.77890 Train accuracy: 96.42% Epoch: 46 (987.8 s): Train Loss: 3.72236 Train accuracy: 96.49% Epoch: 47 (987.7 s): Train Loss: 3.66158 Train accuracy: 96.56% Epoch: 48 (988.9 s): Train Loss: 3.61635 Train accuracy: 96.60% Epoch: 49 (984.1 s): Train Loss: 3.61718 Train accuracy: 96.58% Epoch: 50 (983.7 s): Train Loss: 3.52620 Train accuracy: 96.67% Epoch: 51 (987.2 s): Train Loss: 3.52741 Train accuracy: 96.67% Epoch: 52 (989.7 s): Train Loss: 3.42611 Train accuracy: 96.77% Epoch: 53 (984.0 s): Train Loss: 3.42746 Train accuracy: 96.78% Epoch: 54 (983.6 s): Train Loss: 3.38949 Train accuracy: 96.80% Epoch: 55 (985.9 s): Train Loss: 3.33504 Train accuracy: 96.88% Epoch: 56 (984.4 s): Train Loss: 3.30189 Train accuracy: 96.90% Epoch: 57 (988.7 s): Train Loss: 3.31583 Train accuracy: 96.85% Epoch: 58 (983.8 s): Train Loss: 3.22689 Train accuracy: 96.96% Epoch: 59 (984.7 s): Train Loss: 3.18804 Train accuracy: 97.03% Epoch: 60 (983.7 s): Train Loss: 3.20612 Train accuracy: 96.98% Training finished. Testing... Test Accuracy: 95.94%

The test accuracy is 95.94%, which is a good value for test data.

[didibear]

Good, now launch the predict.py file which use the model in a real environment by taking decision step by step Activate the visualize boolean to see it in action

Also by default the environment is 16x16 grid with observable depth = 5, which is quite a lot of visibility. You can try generating data with higher restrictions : lower depth lower or higher grid size.

[fanyuzeng]

OK, I try it.

[fanyuzeng]

This was a error on gym environment as following: gym.error.UnregisteredEnv: No registered env with id: partially-observable-pathfinding-obstacle-16x16-d5-v0

I google this issue, but can't find the solution.

[didibear]

That the gym_pathfinding library that generate this env, and that's strange because it should generate a it on the fly. This is one of my library, I will look at it, but not now I'm sorry ^^'

[fanyuzeng]

Ok，I change other pathfinding maze to test.