search

reiniscimurs / DRL-robot-navigation

Deep Reinforcement Learning for mobile robot navigation in ROS Gazebo simulator. Using Twin Delayed Deep Deterministic Policy Gradient (TD3) neural network, a robot learns to navigate to a random goal point in a simulated environment while avoiding obstacles.
MIT License
571 stars 119 forks source link

About adding LSTM #9

Closed hjj-666 closed 2 years ago

hjj-666 commented 2 years ago

Hello, I recently tried to use the 180 degree radar data returned each time, first through VAE coding, then through LSTM network, and finally input it into td3 network, but I haven't got any good results. What suggestions can you give

reiniscimurs commented 2 years ago

Hi,

I am not really sure how would I be able to help you with this in any way as you are not giving any specific information about it. I don't think I can help you with purely theoretical extensions of TD3 work.

The only thing I can say is that I don't understand why would you use VAE encoding for laser data in the first place. Clearly for dimensionality reduction, but what kind of result are you expecting to come out of it?

Second, LSTMs are used for time-series data. The implemented TD3 network in this repo assumes independence in the batched data. Meaning, that when a batch is selected for training, it is a single state-action tuple that it is trained on. There is no information about the previous states in the environment, so I am not sure how you would implement LSTM with this repo without significantly rewriting the code.

Please provide more specific implementation descriptions in the future, as it is virtually impossible to answer a question when it is phrased in this manner. Good luck with your implementation and extension.

hjj-666 commented 2 years ago

Unable to register with master node [http://localhost:11311]: master may not be running yet. Will keep trying. Average Reward over 10 Evaluation Episodes, Epoch 19: -156.269393, 1.000000

hjj-666 commented 2 years ago

Average Reward over 10 Evaluation Episodes, Epoch 19: -156.269393, 1.000000 Unable to register with master node [http://localhost:11311]: master may not be running yet. Will keep trying. Recently, when I trained to about 19-23 epoch, there was a problem that I could not connect to the master node. After this problem, the car kept spinning in the simulation and could not enter the next round. What might be the reason

------------------ 原始邮件 ------------------ 发件人: "reiniscimurs/DRL-robot-navigation" @.>; 发送时间: 2021年12月29日(星期三) 凌晨2:31 @.>; @.**@.>; 主题: Re: [reiniscimurs/DRL-robot-navigation] About adding LSTM (Issue #9)

Hi,

I am not really sure how would I be able to help you with this in any way as you are not giving any specific information about it. I don't think I can help you with purely theoretical extensions of TD3 work.

The only thing I can say is that I don't understand why would you use VAE encoding for laser data in the first place. Clearly for dimensionality reduction, but what kind of result are you expecting to come out of it?

Second, LSTMs are used for time-series data. The implemented TD3 network in this repo assumes independence in the batched data. Meaning, that when a batch is selected for training, it is a single state-action tuple that it is trained on. There is no information about the previous states in the environment, so I am not sure how you would implement LSTM with this repo without significantly rewriting the code.

Please provide more specific implementation descriptions in the future, as it is virtually impossible to answer a question when it is phrased in this manner. Good luck with your implementation and extension.

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.Message ID: @.***>