HumanCompatibleAI / imitation

Clean PyTorch implementations of imitation and reward learning algorithms
https://imitation.readthedocs.io/
MIT License
1.27k stars 240 forks source link

Contributing VIPER for explainable IL #667

Open faizankshaikh opened 1 year ago

faizankshaikh commented 1 year ago

I aim to implement VIPER as part of my project using this library, with a long term goal of contribution. As I am still new to code contributions, I would gladly request your suggestions throughout the way.

AFAIK, VIPER is just DAGGER, but BC is replaced by Decision Trees. Therefore all I need to do (correct me if I am wrong) is to build a algo_base.DemonstrationAlgorithm compatible DT using sklearn.

Let me know if this is something worth integrating with the library

AdamGleave commented 1 year ago

Thanks for the interest in contributing! VIPER would be a nice algorithm to have working with imitation. My suggestion would be to initially develop it in a separate repo but building on top of imitation (and perhaps @ernestum can help with some code review?) and then we can decide later whether to merge it into the main repo. Long-term we should consider having an imitation-contrib repo similar to with Stable Baselines.

Our current DAgger implementation is quite tightly coupled with bc.BC so you may need to change that -- definitely happy to have PRs that refactor DAgger to make it more general.

ernestum commented 1 year ago

I am happy to help out! I think this is a great opportunity to check how accessible the imitation library is for newcomers and to refactor DAgger. @faizankshaikh we can discuss an initial design for your implementation and that discussion might reveal some ways in which we can improve the DAgger implementation to make it easier for you to implement VIPER. Feel free to contact me via email so we can schedule a call.

faizankshaikh commented 1 year ago

That works, thanks!