search

knowledgedefinednetworking / DRL-GNN

BSD 3-Clause "New" or "Revised" License
200 stars 35 forks source link

About using OpenAI Gym to implement the network topology environment #4

Closed Alilalily closed 3 years ago

Alilalily commented 3 years ago

Hi, I am a student Interested in your team work about using GNN to represent the computer network architecture and agent's action. After reading this paper, I noticed that the network topology environment in this paper was implemented using the OpenAI Gym framework. Did you using the ns3-ai to finish the environment ? Or using the original OpenAI Gym?

paulalmasan commented 3 years ago

Hi, in the implementation we used only the OpenAI Gym toolkit to develop the environment of the DRL framework. Specifically, we used the networkx (https://networkx.org/) library to define the network graph topology within the environment.

Alilalily commented 3 years ago

Thanks for your reply. I am currently doing the experiments in this paper, but the environment is not using Open AI. Now I have a problem, how to determine the link available bandwidth? My idea is to build an SDN network topology in Mininet to obtain it in real time, but it seems difficult to realize it. May I ask how to determine the value of available bandwidth of the link in your paper

paulalmasan commented 3 years ago

Hi, if you integrate Mininet to obtain the link utilization in real time you should take into account the training time. The reason is that the DRL training process is computationally intensive and it takes quite some time to train the DRL agent even in a small simple scenario. In our paper, we add the value of the traffic demand to the affected links to obtain the link utilization, which is straightforward. Adding Mininet would make this process even slower. Another alternative would be to create a dataset with topology+link features+traffic matrix and train the DRL using the dataset. The dataset can be obtained using simulation or real world topologies. However, in this way the DRL setup should be re-designed. I hope this clarifies your doubts :)

Alilalily commented 3 years ago

Thank you so much!In fact, we used Mininet simulation to obtain rewards, which took a long time. Later, we used Mininet simulation to obtain data sets for model training. Your reply really solved my doubts.