search

CN-UPB / NFVdeep

NFVdeep: Deep Reinforcement Learning for Online Orchestration of Service Function Chains
MIT License
55 stars 14 forks source link

need a help #2

Closed phongvu0403 closed 3 years ago

phongvu0403 commented 3 years ago

Hello, thank you for this very wonderful project. This is also what I was looking for for my graduate project. I have a few questions you can answer these are: -In the update function in the network.py file, when the SFC exceeds their TTL, will the environment be changed? For example, those SFCs return CPU, Memory, Bandwidth resources to the nodes that the SFCs have embedded into. -About results: I understand an episode is a set of states from the beginning to the last (from the first SFC that joins the system until the last SFC leaves the system). In the following episodes, the agent is better trained based on the experience gained from the previous episodes. Do I understand that correctly? Thanks in advance!!!

stefanbschneider commented 3 years ago

-In the update function in the network.py file, when the SFC exceeds their TTL, will the environment be changed? For example, those SFCs return CPU, Memory, Bandwidth resources to the nodes that the SFCs have embedded into.

Yes, all blocked resources are released and freed up when an SFC exceeds its TTL and leaves the network. The same goes if an SFC request is rejected - no resources are blocked.

-About results: I understand an episode is a set of states from the beginning to the last (from the first SFC that joins the system until the last SFC leaves the system). In the following episodes, the agent is better trained based on the experience gained from the previous episodes. Do I understand that correctly?

Yes, exactly. In this setting, there's no really natural definition of an episode, but most RL approaches are still focused on the episodic rather than continuous setting. Hence, we defined an episode is as a given number of SFC request arrivals. The agent learns during the episode and continues to use and improve its learned policy in the following episodes.

I hope this answers your questions?

@stwerner97 @NilsLuca Please correct me if I'm wrong.

phongvu0403 commented 3 years ago

Thank you for this answer. May I ask one more question: When I assign a specific SFC to the system, how do I know which VNFs in that SFC are embedded in which respective node. Thanh you again!

stefanbschneider commented 3 years ago

Good question. To be honest, I'm not sure.

@stwerner97 @NilsLuca Could one of you answer this? In the example results, I couldn't find any information on the actual placement; surely that's recorded somewhere? Could you also add 1-2 sentences to the Readme, describing the result format, eg, how/where to find the computed placement/embedding? Thanks!

stwerner97 commented 3 years ago

If I recall correctly, VNF placements are recorded only internally in env.sfc_embedding which maps SFC objects to lists of placement decisions (node indices) but removes them once their TTL passed. If you want to add embeddings to our logging class NFVDeepMonitor extending its step method would likely be the easiest way. If you are interested I can look into it at the end of the month 👍

phongvu0403 commented 3 years ago

Thank you very much! I am really interested in this problem. I will wait until the end of the month.

stwerner97 commented 3 years ago

Tabulating of placement decisions added in #3.

stefanbschneider commented 3 years ago

@stwerner97 Nice! Could you make a small example to illustrate what that means for using NFVdeep and retrieving its results?