Algorithm checklist 1

[ajaysub110]

As discusses, let's start off with some popular algorithms in each domain not necessarily in this order. Feel free to add and update this when necessary.

1. Deep RL

   • [x] Vanilla Policy Gradient
   • [x] PPO1
   • [x] DDPG
   • [ ] DQN (all variants as options for a single model)
   • [x] SAC
   • [x] TD3

2. Classical RL

   • [x] UCB-1
   • [x] Thompson Sampling
   • [x] SARSA (with eligibility traces)

[ajaysub110]

I'll work on HER

[Sharad24]

It would be great if we can log results from algorithm implementations. That way, we dont have to run them again later on

[ajaysub110]

Yeah. Can you create a spreadsheet where we add results (mean and variance of rewards) on different environments?

[Sharad24]

We can probably have a markdown file with a table, etc inside dir deeprl/ no?

[ajaysub110]

Yeah makes sense. Should we also create a json/yml file for hyperparameters?

[Sharad24]

Yeah. We can add these details to the Contributing Guidelines I think

On 31-Mar-2020, at 8:19 PM, Ajay Subramanian notifications@github.com wrote:

Yeah makes sense. Should we also create a json/yml file for hyperparameters?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SforAiDl/JigglypuffRL/issues/6#issuecomment-606675273, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH72FJ4TUXUAKC6K3TNIX5DRKH7IVANCNFSM4LWRS65A.

[ajaysub110]

Ok, will add that once these files have been made

[Sharad24]

@mehulrastogi Can you work on adding any evolutionary algorithms?

[mehulrastogi]

For sure sharad!

On Tue, Mar 31, 2020 at 1:17 PM Sharad Chitlangia notifications@github.com wrote:

@mehulrastogi https://github.com/mehulrastogi Can you work on adding any evolutionary algorithms?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SforAiDl/JigglypuffRL/issues/6#issuecomment-606760708, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFINXC32FSYD4MNMXBFEXJ3RKIQSHANCNFSM4LWRS65A .

[Sharad24]

@mehulrastogi Could you please create an issue on the same so that we can discuss basic structure, etc beforehand?

[mehulrastogi]

Okay, so there are many Evolutionary Algorithms. I guess you want to cover the basic and the popular ones right? so those can be listed as follows:-

• Basic ES (https://openai.com/blog/evolution-strategies/)
• NEAT
• HyperNEAT
• Many more are there. You can look at link

Although this is not an exhaustive list at all the literature regarding Genetic/evolutionary algorithms can be very algorithmic and problem-specific.

Would also like to point people looking into this towards link for the difference between GA and ES.

[Sharad24]

We can start off with NEAT, ES algos here and HyperNEAT (in any order). These look like the main ones for now, we can keep adding stuff later on. What do you think?

[Sharad24]

Of course, we should focus on the ones which are reproducible+benchmarkable first, otherwise we might just be stuck with code no ones gonna look at

[mehulrastogi]

Of course, we should focus on the ones which are reproducible+benchmarkable first, otherwise we might just be stuck with code no ones gonna look at

I think to focus on NEAT and HyperNEAT first. Of course you can find the implementation for NEAT for python here. If you want NEAT implementation in PyTorch you can find them here. After replicating the benchmark for NEAT for your tasks you will want to focus on HyperNEAT and then the Deep Neuroevolution algorithms by UBER. In my opinion Random Search -> Basic ES -> NEAT -> HyperNEAT -> DeepNeuroevolution.

[Sharad24]

Ok, thanks for the info!

I was hoping you could take some of these on? If not, thats ok, we will come back to it later after getting some core details of deeprl sorted.

On 08-Apr-2020, at 12:48 AM, Mehul Rastogi notifications@github.com wrote:

Of course, we should focus on the ones which are reproducible+benchmarkable first, otherwise we might just be stuck with code no ones gonna look at

I think to focus on NEAT and HyperNEAT first. Of course you can find the implementation for NEAT for python here https://neat-python.readthedocs.io/en/latest/. If you want NEAT implementation in PyTorch you can find them here https://github.com/uber-research/PyTorch-NEAT. After replicating the benchmark for NEAT for your tasks you will want to focus on HyperNEAT and then the Deep Neuroevolution algorithms by UBER. In my opinion Random Search -> Basic ES -> NEAT -> HyperNEAT -> DeepNeuroevolution.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SforAiDl/JigglypuffRL/issues/6#issuecomment-610572301, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH72FJYU327C7GT6LT75UQDRLN4AJANCNFSM4LWRS65A.

[Sharad24]

Closing this in favour of #101 as thats the only one left