search

jjzgeeks / FL-DLT3

Exploring Deep Reinforcement Learning-Assisted Federated Learning for Online Resource Allocation in Privacy-Preserving EdgeIoT
MIT License
18 stars 2 forks source link

问题 #3

Open vincent-NJW opened 4 days ago

vincent-NJW commented 4 days ago 
 Hello, I am a graduate student in China. My research direction is the application of reinforcement learning in edge computing and federated learning. Recently, I carefully read your article "Exploring Deep Reinforcement Learning Assisted Federated Learning for Online Resource Allocation in Privacy Resilient EdgeIoT" and found that your idea is very excellent and forward thinking. Then, I ran the project code you uploaded to Github, but encountered some problems. After 1000 episodes, I found that the reward for each round is None.May I ask for specific reasons. Thank you, hope reply.
jjzgeeks commented 3 days ago 
 Hello, I am a graduate student in China. My research direction is the application of reinforcement learning in edge computing and federated learning. Recently, I carefully read your article "Exploring Deep Reinforcement Learning Assisted Federated Learning for Online Resource Allocation in Privacy Resilient EdgeIoT" and found that your idea is very excellent and forward thinking. Then, I ran the project code you uploaded to Github, but encountered some problems. After 1000 episodes, I found that the reward for each round is None.May I ask for specific reasons. Thank you, hope reply.

Hi, I have never encountered this problem. Considering the problem you are facing, it is recommended to focus on only one episode, make sure you have input data, and your actions fall between lower bound and upper bound.

Good luck, Jingjing

vincent-NJW commented 3 days ago

Thank you very much for your reply. I ran the jingjing_td3_lstm_datasize.py file, and when creating the environment, I passed in the original_data_1000_20_6_mormal. mat file that you submitted. I think it should be an environmental issue. Do you think my above running process is standardized.

jjzgeeks commented 3 days ago

Thank you very much for your reply. I ran the jingjing_td3_lstm_datasize.py file, and when creating the environment, I passed in the original_data_1000_20_6_mormal. mat file that you submitted. I think it should be an environmental issue. Do you think my above running process is standardized.

Hi, I strongly recommend that you run jingjing_td3_lstm_v8.py file first. The jingjing_td3_lstm_datasize.py is not the mian file.

Best, Jingjing

vincent-NJW commented 3 days ago

Thank you very much for your reply. I ran the jingjing_td3_lstm_datasize.py file, and when creating the environment, I passed in the original_data_1000_20_6_mormal. mat file that you submitted. I think it should be an environmental issue. Do you think my above running process is standardized.

Hi, I strongly recommend that you run jingjing_td3_lstm_v8.py file first. The jingjing_td3_lstm_datasize.py is not the mian file.

Best, Jingjing

  Sure, thank you very much for your suggestion. I will continue to delve into the research over the next few days. If I have any questions later, may I consult you? Additionally, can I contact you through the email address provided in your paper?
jjzgeeks commented 2 days ago

Thank you very much for your reply. I ran the jingjing_td3_lstm_datasize.py file, and when creating the environment, I passed in the original_data_1000_20_6_mormal. mat file that you submitted. I think it should be an environmental issue. Do you think my above running process is standardized.

Hi, I strongly recommend that you run jingjing_td3_lstm_v8.py file first. The jingjing_td3_lstm_datasize.py is not the mian file. Best, Jingjing

  Sure, thank you very much for your suggestion. I will continue to delve into the research over the next few days. If I have any questions later, may I consult you? Additionally, can I contact you through the email address provided in your paper?

No problem. Feel free to contact me if you have any questions.

Good luck, Jingjing

vincent-NJW commented 2 days ago

Hi there,

I ran the jingjing_td3_lstm_v8.py file from your repository. After 1000 episodes, I checked the rewards and saw that the reward curve didn't go up; it actually went down a bit. Is this normal? I didn't change any parameters and just used the default settings.Also, can I contact you through your email?

Thanks!

vincent-NJW commented 1 day ago

Oh, sorry, I just checked the simulation part of your paper, and it indeed shows this phenomenon. Your AE gain remains at a relatively stable value. Are you referring to the reward value in the reinforcement learning algorithm when you mention AE gain? This is the first time I've encountered a situation where the reward value doesn't increase significantly

vincent-NJW commented 1 day ago

I modified the generate_data_10.py file to generate an environment data file with 5000 steps for 60 clients. Then, I changed max_episode=5000 in jingjing_td3_lstm_v8.py to see if the reward increases. It is currently running.

jjzgeeks commented 1 day ago

Oh, sorry, I just checked the simulation part of your paper, and it indeed shows this phenomenon. Your AE gain remains at a relatively stable value. Are you referring to the reward value in the reinforcement learning algorithm when you mention AE gain? This is the first time I've encountered a situation where the reward value doesn't increase significantly

Hi, the displayed AE gain is just in each communication round instead of accumulated rewards over the communication rounds.

Best, Jingjing