search

hsahovic / poke-env

A python interface for training Reinforcement Learning bots to battle on pokemon showdown
https://poke-env.readthedocs.io/
MIT License
290 stars 98 forks source link

Compatibility/examples/help using tf-agents #198

Open Benjamin-Etheredge opened 2 years ago

Benjamin-Etheredge commented 2 years ago

Would it be possible to get some examples up using tf-agents?

I'm attempting to leverage poke-env to drive some RL agents. keras-rl2 seems to be discontinued, so TF Agents seems like the best route.

I'm having a lot of trouble adapting the player-env to working with tf-agents or other gym-consuming tools. Is there a nice way to just create a repeating environment that constantly goes up against an opponent (e.g. PlayerEnv that has step and rest but always goes up against the same opponent and resets when the battle is over?

I'm attempting to adapt play_against, but it seems overly complex to pass in functions and wrappers to make it work.

Any help would be greatly appreciated. I love the work you've done thus far.

Benjamin-Etheredge commented 2 years ago

I manage to get the wrapping code working. It was more elegant than I thought. My issue was it didn't play nicely in a jupyter notebook. The example code would crash due to "no active battles" if run in a jupyter notebook.

hsahovic commented 2 years ago

Hey @Benjamin-Etheredge,

Thanks for bringing this up. If I understand correctly, you got your code to run? I'll run some tests with jupyter and see if I can do something about it. If not, I'll update the docs / readme.

Benjamin-Etheredge commented 2 years ago

Yes, I have gotten it working. I still think it would be helpful to have the gym-compatible environment available outside the wrapper. Is it possible to handle the wrapping logic with some sort of automatic setup and tear-down? Or maybe switching from wrapping a function to a context manager? I'm not sure what the best way to do it would be. I'm sure I'll have a better idea after playing with it more.

arcaputo3 commented 2 years ago

@Benjamin-Etheredge would you mind sharing this notebook in your forked examples? Really love this idea as I've had issues with keras-rl2 and the fact that it's no longer maintained.

hsahovic commented 2 years ago

@Benjamin-Etheredge I'd happily add examples to the docs if you have a minimally running rl example that uses another framework!

Benjamin-Etheredge commented 2 years ago

So the work going on to rework the gym wrapper would make this environment work with any other algorithms. I've got it working with the PPO example from pytorch lightning right now.

I've got a minimal viable conversion of the PlayerEnv to work with gym over at #213. With it, any example code that uses gym.make(..) would work. I'm still playing around with how to make it better.

EllangoK commented 2 years ago

@Benjamin-Etheredge I would also like to ask if you still have the pytorch or whatever version you made and would like available as I have had difficulties with the existing examples.

DiTo97 commented 1 year ago

Sorry for waking the dead, but I just recently came back to this repository.

Out of curiosity, @Benjamin-Etheredge, did you menage to integrate the Gym library (now Gymnasium) with Poke-env? I noticed that #213 was closed, but it does not seem to have been merged in the code base.