search

facebookresearch / mbrl-lib

Library for Model Based RL
MIT License
952 stars 154 forks source link

[Feature Request] Upload Dynamics Models to the HuggingFace Hub #169

Closed natolambert closed 1 year ago

natolambert commented 1 year ago

🚀 Feature Request

Add functionality to upload dynamics models /policies to the HF hug at end of training or during training for sharing / fine-tuning.

This would like like

model.from_pretrained("mbrl/cheetah.bin")
model.save_pretrained("mbrl/hopper.bin")

Motivation

We want to be able to re-use computation and make easier demo's showcasing this library.

Happy to help with this.

Additional context

Add any other context or screenshots about the feature request here.

luisenp commented 1 year ago

Hi @natolambert, sorry for the delay, it was a long weekend :)

This definitely sounds interesting, but I don't have any experience doing this. If you are willing to get a PR started, that would be awesome!

natolambert commented 1 year ago

Great. Happy to take the lead. May grab some co-workers to help.

This was motivated by a new workshop @agarwl is making to re-use computation in RL 🤗

natolambert commented 1 year ago

@luisenp do you have checkpoints of the models used in the benchmarking efforts of the paper? Or do I have to re-train to get them. Thanks for checking!

(those will be my proof of concept for this feature!)

luisenp commented 1 year ago

I have a lot of checkpoints stored for different versions of the library, but need to dig a little bit to check if any correspond to the latest version. I can also run some jobs again if need be. Would you like to start with something in particular?

natolambert commented 1 year ago

Generally my goal for this PR would be do get the code working (actually the easy part), and say you can easily run a couple agents that are pretrained with xyz lines of code.

Then, if we have some pretrained models, it's also nice for people to new research ideas with them.

So, I can work with whatever tasks / models you have as long as we can document them. Don't need a ton to get started, we just want to use them to debug the code I write!

agarwl commented 1 year ago

The in-person workshop on reincarnating RL (reusing prior computation) also got accepted at ICLR 2023 -- this might be useful there.

natolambert commented 1 year ago

Closed with #178