search

RealVNF / distributed-drl-coordination

Distributed Online Service Coordination Using Deep Reinforcement Learning
19 stars 6 forks source link

Migration to SB3 #7

Open katjamuench opened 1 year ago

katjamuench commented 1 year ago

Are there plans to move this project to SB3 eventually?

stefanbschneider commented 1 year ago

Hi @katjamuench , there are currently no plans of migrating. I only have time to respond to questions and fix minor bugs, but not for larger efforts or refactoring I am afraid.

If you or anyone else is willing to migrate, I would be happy to receive and review the PR.

burnCalories commented 1 year ago

Hi @stefanbschneider . I have completed the migration of SB3 before, and all I need to do is download some software packages and modify some code changes to complete the migration. Do you need me to publish my code or create a branch? I am willing to provide assistance.

katjamuench commented 1 year ago

I've also migrated it by now, however, I did need to modify the setup requirements for coord-sim and common-utils, though this seems to have made no significant impact on performance. I do have about a five percent decline in successful flows, but I assume that is because I've just tried using PPO with no parameters adjusted, since SB3 doesn't offer ACKTR.

stefanbschneider commented 1 year ago

If you have a fork using SB3, I'd be happy to link it in the Readme!

Since @katjamuench reported that the results are a bit worse and there's no ACKTR in SB3, I'd rather keep this repo as is. It is supposed to represent the corresponding paper, where we produced all results with ACKTR.