search

SatCom-TELMA / MA-DRL_Routing_Simulator

Multi-Agent Deep Reinforcement Learning (MA-DRL) Routing Simulator for satellite networks
35 stars 7 forks source link

Question about SimulationRL.py not running correctly #3

Closed HansJack1999 closed 2 months ago

HansJack1999 commented 2 months ago

Hello, I am trying to run the SimulationRL.py file, but I encountered an issue where the output seems to loop endlessly:

Saving ISLs map...
----------------------------
c:\Users\xxx\Desktop\MA-DRL_Routing_Simulator-main\SimulationRL.py:1024: RuntimeWarning: divide by zero encountered in double_scalars
  timeToSend = sendBuffer[1][0].size / destination[2]
Matching: Greedy
Removing duplicated links...
Establishing remaining edges...
----------------------------------
Matching: Greedy
Removing duplicated links...
Establishing remaining edges...
c:\Users\xxx\Desktop\MA-DRL_Routing_Simulator-main\SimulationRL.py:6394: RuntimeWarning: divide by zero encountered in double_scalars
  dataRate=1
----------------------------------
c:\Users\xxx\Desktop\MA-DRL_Routing_Simulator-main\SimulationRL.py:1024: RuntimeWarning: divide by zero encountered in double_scalars
  timeToSend = sendBuffer[1][0].size / destination[2]
Matching: Greedy
Removing duplicated links...
Establishing remaining edges...
c:\Users\xxx\Desktop\MA-DRL_Routing_Simulator-main\SimulationRL.py:6394: RuntimeWarning: divide by zero encountered in double_scalars
  dataRate=1
----------------------------------
c:\Users\xxx\Desktop\MA-DRL_Routing_Simulator-main\SimulationRL.py:1024: RuntimeWarning: divide by zero encountered in double_scalars
  timeToSend = sendBuffer[1][0].size / destination[2]
ERROR! Sat 19_19 tried to send block to 18_17 but did not have it in its linked satellite list

I noticed that in the README.md, it is mentioned that "The updateSatelliteProcessesRL() method in the SimulationRL.py file does not work correctly". Could this be related to the issue I am experiencing? Is there any workaround or fix available to run the simulation properly?

Looking forward to your guidance. Thank you!

Fedeloz commented 2 months ago

Hi, thank you for reaching out!

The "infinite loop" is not an infinite loop actually. I noticed that in the default settings, in "inputRL.csv" the test length was set to 20 seconds, and the movementTime in "SimulationRL.py" is set to 0.05s. This means that the satellites position are being update every 0.05 seconds, for 20 seconds, which is a lot of times and can look infinite. Every time the satellites move they have to match again with their neighbors and create the inter satellite links again.

I updated the code, where the test length is only 2 seconds, the update frequency "movementTime= 10" seconds, which means the constellation will not move at all, since the update frequency is higher than the total test length. I also set the Kepler constellation as default, which will make the simulation go faster since Starlink is very dense and more computation needs to be done.

I pushed a new version where, just by running "SimulationRL.py" you will see how the algorithm learns to find the optimal path between the two first locations in "inputRL.csv", Malaga and Los Angeles. You can change the locations by changing the order of this list, or add more locations by increasing "GTs" variable.

With the current settings, the complete simulation takes around 12-15 min on my mac m2 chip.

Please, let me know if everything works for you now, and contact again for any additional issue and/or question.

HansJack1999 commented 2 months ago

Thank you so much for your detailed response and insights!

I appreciate your clarification regarding the issue of the "infinite loop." It turns out, after a bit of step-by-step debugging on my end, that the simulation wasn't in an infinite loop, but was simply running very slowly due to the large size of the simulated constellation. This made it difficult to observe any progress. In fact, after nearly 12 hours of simulation, I had only completed about 15% (3 seconds).

Your response was incredibly helpful in confirming that the system was running normally and that the slow progress was not due to any unknown issue with my environment's version. This reassurance has significantly aided my understanding of the project.

Thank you once again for your helpful and informative reply!

Best regards