Feasibility in Windows

[MarleneBs]

Hallo :) I have two questions, because I would like to run the code on a Windows PC and I have not yet managed to simply replace reverb. Are there any ideas, suggestions or approaches on how I can run the code on a Windows PC? And are there any approaches to replacing reverb without too much effort? Maybe someone has a simple idea, thank you in advance. Many thanks, regards :)

[jtscs]

Hello, I have looked a bit into the tensorflow documentation and there seems to be only one replay bufffer that does not depend on reverb, the TFUniformReplcayBuffer (https://www.tensorflow.org/agents/api_docs/python/tf_agents/replay_buffers/TFUniformReplayBuffer). There is one example where it is used here https://github.com/tensorflow/agents/blob/master/docs/tutorials/9_c51_tutorial.ipynb. Currently I can only offer the option to run the jupyter notebook version of the centralized approach (https://github.com/jtscs/DRL_volt-var_control/tree/main/Jupyter_Notebook_Version) on google colab or so. I will try to find some time in the next couple days to see if the TFUniformReplcayBuffer can be used and offer the possibility to run the repo without reverb. Kind Regards :)

[MarleneBs]

Thank you very much! In that case, I will continue working on the approach with the TF Uniform Buffer. It seems to me that it has some differences or less functionality compared to the Reverb Buffer in some places. So, I will start by trying to replace Reverb step by step with the TF Uniform Buffer.

Thank you for checking and finding an example, I appreciate your effort. If you have any other or final suggestions for achieving this on Windows, please let me know :)

It still seems to me that we mainly need to change the Replay Buffer, what do you think?

Thanks and kind regards:)

[jtscs]

Hi, not sure if you already made a working version for windows, otherwise I finished a version that can run under windows by using the tf uniform buffer. You can access it in the windows-version_without-reverb branch. Also feel free to open merge requests if you find code improvements or other run options :) So far I only adjusted the centralized training but the changes should be similar for the decentralized training, maybe I will find some time soon to change that too.

[MarleneBs]

Thank you very much! I had it almost, but not yet completely, so I thank you very much for the effort and will now take a look at the corresponding branch.