search

trunk96 / wireless-network-simulator-v2

Wireless Network Simulator version 2.0
20 stars 9 forks source link

hello,I have a question for wireless-network-simulator-v2 #2

Open 787114137 opened 5 months ago

787114137 commented 5 months ago

In the lexicographicqlearning.py file in your text_gym file, there are constraints when defining objects, and what do constraints represent? This is the “constraints” in the first line of the code below

`def init(self, env, model_name, constraints):

    self.env = env
    self.model_name = model_name
    action_size = self.env.action_space.n
    state_size = self.env.observation_space.n
    constraint_size = len(constraints)
    self.constraints = constraints
    self.qtable_constraints = np.zeros((constraint_size, state_size, action_size))
    self.qtable = np.zeros((state_size, action_size))
    self.avg_time = 0
    self.steps_per_episode = 1000
    self.gamma = None`

I hope you can help me, thank you very much

787114137 commented 5 months ago

For your convenience, you wrote: learner = lexicographicqlearning.LexicographicQTableLearner(env, "CAC_Env", [0.075, 0.10, 0.15])

trunk96 commented 5 months ago

Hello, the constraints are made to limit the possible actions that an agent may do. Indeed, if in the current state the value of an action is greater than the constraint, this action cannot be selected by the agent. This is made so to not violate constraints for other agents, which are of higher priority.

To understand the concept of Lexicographic RL, I suggest you to give a look on Google Scholar and/or here https://www.ijcai.org/proceedings/2022/0476.pdf

Il giorno sab 23 mar 2024 alle ore 08:29 787114137 @.***> ha scritto:

For your convenience, you wrote: learner = lexicographicqlearning.LexicographicQTableLearner(env, "CAC_Env", [0.075, 0.10, 0.15])

— Reply to this email directly, view it on GitHub https://github.com/trunk96/wireless-network-simulator-v2/issues/2#issuecomment-2016394651, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGG5EGE36POH2UKIAHEFIYLYZUVM3AVCNFSM6AAAAABFEOCJU2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJWGM4TINRVGE . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- [image: Logo] https://www.diag.uniroma1.it/ Emanuele De Santis Assistant Professor (RTD-A) in Automatica email: @.*** Room A215 Via Ariosto, 25 - 00185 Roma INFORMATIVA SULLA PRIVACY Ai sensi del D.Lgs. 196/2003, si precisa che le informazioni contenute in questo messaggio e i suoi eventuali allegati sono riservate e per uso esclusivo del destinatario. E' vietata la copia o la distribuzione del messaggio a terzi. Chiunque riceva questo messaggio per errore, è pregato di eliminarlo. CONFIDENTIALITY NOTICE Pursuant to Legislative Decree No. 196/2003, you are hereby informed that this message contains confidential information intended only for the use of the addressee. If you are not the addressee and have received this message by mistake, please delete it. You may not copy or disseminate this message to anyone. Thank you.

787114137 commented 5 months ago

Thank you for your reply, as I am a Chinese university student, my language organization may not be clear. Regarding this question, I would like to ask on what basis the values of these constraints are calculated. I have reviewed your relevant papers and have not found any definitions regarding these constraints. I guess these constraints may refer to one of resource constraints, performance constraints, security constraints, or others. Sorry to bother you