Simulation problem

[hjj-666]

Hello, I am also a person who is doing the autonomous exploration of this robot. When can you publish your code? I recently encountered a problem in gym-gazebo simulation. Can I talk to you? My email is 958901886@qq.com

[reiniscimurs]

Hi,

I am not sure if I will be able to post the RL training code anytime soon, but I will see what I can do about it once I have more time. But the code should work with any "hard-coded" navigation behaviors at least for testing purposes. You just need to send an action (linear and angular velocity) to the GDAM_env.py file. That being said, the code is not meant for implementation in Gazebo simulator, but rather for real implementation with Pioneer mobile robot platform in real environment and real sensors. Though, I am sure with tweaking it can run in the simulator as well.

If you have any specific issues with running the code, I think the best solution would be to post it here. Let me know what your specific issue is and I will try to help as much as I can.

[hjj-666]

[hjj-666]

I have created such an environment at present. I initialized the initial point and the target point randomly, and gave a reward function based on the distance to the target point, but it just can’t converge. How did you solve the convergence problem?

[reiniscimurs]

Is this regarding the training of the neural network itself? What network did you use and what parameters? How do you update the neural network parameters and what do you use and network inputs?

[hjj-666]

The network currently in use is the dqn network officially provided by gym, and there is no major change. The changes are mainly in the environment, reward function, setting the initial point and target point. The input of the network is 270 degree 100-line lidar data

[reiniscimurs]

Do you have a link to the code? As far as I know the gym DQN works with images and uses 2D convolution. You wouldn't need convolution for a 2D laser data, but I don't know your implementation. Another thing I would suggest is the 270 degree 100 line lidar data seems like a lot. I would try reducing that by either using sparser readings or "bagging" the laser data. You wouldn't need more than 10 laser directions (10 dimension laser state) to check if your code converges.

[hjj-666]

Thank you very much, I will make improvements based on your suggestions

[reiniscimurs]

I have made a version of the training code available on github at https://github.com/reiniscimurs/DRL-robot-navigation Might want to look at that. Be warned, that I have not yet had the time to thoroughly test the repository. It works on my system but all the dependencies are already met there so I cannot guarantee that it works out of the box on any other system.

[hjj-666]

Thank you very much for the simulation code, I think your code must be of great help to me