An error occurred when I executed this code

Dronie / D2D_A2C

Channel Selection and Power Control for D2D Communication via Online Reinforcement Learning

35 stars 15 forks source link

An error occurred when I executed this code #2

Open Gavin-cy opened 3 years ago

Gavin-cy commented 3 years ago

        self.Critic_return, self.advantage = trfl.sequence_advantage_critic_loss(self.baseline_,
            self.reward_, self.discount_, self.bootstrap_, lambda_=lambda_, 
            baseline_cost=baseline_cost)

      AttributeError: module 'trfl' has no attribute 'sequence_advantage_critic_loss'

     This problem has been bothering me, can you help me solve this problem？

Dronie commented 3 years ago

I think this error is because I edited one of the files in the trfl library. Here is the edited file: https://github.com/Dronie/D2D_A2C/blob/master/discrete_policy_gradient_ops.py just replace with the original (it shouldn't cause anything else to break otherwise)

Let me know of any other issues (I can't remember if I changed anything else)

Gavin-cy commented 3 years ago

Thank you for answering my question！

after I modified the code in the source file, this error occurred when I ran it. AttributeError: module 'trfl.base_ops' has no attribute 'LossOutputNoExtra'.Did you modify the code of this file？

Dronie commented 3 years ago

I think I did yes, I have uploaded my version here: https://github.com/Dronie/D2D_A2C/blob/master/base_ops.py

Gavin-cy commented 3 years ago

I'm truly grateful for your help. The code ran successfully.Your code is too helpful，I want to know some derivations about the code，Could you provide some relevant papers on these code files

Dronie commented 3 years ago

D2D_env_discrete.py

This code was provided by my supervisors so I don't have any documentation for it unfortunately

base_ops.py, discrete_policy_gradient_ops.py

This is code I modified from DeepMind's TRFL library (https://github.com/deepmind/trfl)

IAC.py

This is an implementation of a standard, single agent actor-critic with advantage as a baseline (Should be in most RL books)

MHIAC.py, dual_critic.py, dual_critic_partially_centralized.py, coma_centralized.py, coma_partially_centralized.py

These are my own work (some are minor alterations of existing ideas)
Descriptions for them are in my thesis (https://github.com/Dronie/D2D_A2C/blob/master/thesis.pdf)
COMA paper (https://arxiv.org/abs/1705.08926)

Hope this helps

Gavin-cy commented 3 years ago

Your help the other day was greatly appreciated!

youshiyin commented 1 year ago

Hello, I have recently been reproducing this code, encountered some problems, can you add your WeChat and discuss with you

AenaKim commented 6 months ago

你好，最近在复现这段代码，遇到了一些问题，可以加你的微信和你讨论一下吗

你好，请问你还在做这个实验吗