fruffy
commented
2 years ago Yes, indeed. This is intentional. The algorithm is interfering actively with TCP's congestion control. There is not necessarily a data race as qdiscs are not really actively modifying the CWND. The idea behind this was to see whether you could run an RL algorithm on top of TCP and optimize its typically bursty behavior.
The implementation is very rudimentary, so there are surely better ways to do this. For example, by exploiting TCP-BPF: https://legacy.netdevconf.info/2.2/papers/brakmo-tcpbpf-talk.pdf
Hi, I'm currently studying your code and I found your control pipeline working as the flow chart below. The node controller will start during the initalization of the environment and it will keep working as the pattern shown in the figure.
Please correct me if I'm wrong.
However, I'm just wondering while deploying rl algorithm like appo, ddpg, will TCP congestion control still work in the background? If so, will there be a confliction between function _rtnl_qdisc_tbf_set_rate(ctrl_handle, (uint32_t) txrate, burst , 0); and TCP congestion control? Because according to my understanding, the essence of change the sending rate under tcp is changing the size of CWND. If both algorithms are doing the same thing, there seems to be a data race.
Thanks for your time for reading my issue. I'd appreciate it if you could give me some help.