search

farismismar / Q-Learning-Power-Control

Code for the following publication: F. B. Mismar, J. Choi, and B. L. Evans, "A Framework for Automated Cellular Network Tuning with Reinforcement Learning", IEEE Transactions on Communications, vol. 67, no. 10, Oct. 2019, pp. 7152-7167, DOI 10.1109/TCOMM.2019.2926715.
44 stars 28 forks source link

How to switch deep environment to tabular environment? #3

Open bszeng opened 3 years ago

bszeng commented 3 years ago

Hi, I had some trouble when I was planning to simulate Q-Learning Algorithm for VoLTE Closed Loop Power Control in Indoor Small Cells. In order to switch to tabular environment, I did the following:

  1. Change the agent class 
    from environment import radio_environment
#from DQNLearningAgent import DQNLearningAgent as QLearner # Deep with GPU and CPU fallback
from QLearningAgent import QLearningAgent as QLearner
  2. Call the tabular function 
    #    run_agent_fpa(env)
run_agent_tabular(env)
#    run_agent_deep(env)

    I went through the code, but I didn't find a way to switch environments:

    > Ep. | TS | Recv. SINR (srv) | Recv. SINR (int) | Serv. Tx Pwr | Int. Tx Pwr | Reward 
> ------------------------------------------------------------------------------------------------------------
> [-311.94796062  325.77365607  617.88366191 -146.98785931  2.94720088  13.78845061]    
> Traceback (most recent call last):
>   File "main_modify.py", line 592, in <module>
>     run_agent_tabular(env)
>   File "main_modify.py", line 56, in run_agent_tabular
>     action = agent.begin_episode(observation)
>   File "/home/nemo/workspace/Q-Learning-Power-Control/voice/QLearningAgent.py", line 52, in begin_episode
>     self.state = observation + np.zeros(self.state_size, dtype=int)
> ValueError: operands could not be broadcast together with shapes (6,) (3,)

Does anyone know what to do with it?

s-omid commented 2 years ago

Hi, I have the same problem, did you find a solution to this?