search

google-deepmind / acme

A library of reinforcement learning components and agents
Apache License 2.0
3.52k stars 427 forks source link

Does this support pytorch-based agents? #20

Closed drozzy closed 4 years ago

drozzy commented 4 years ago

Hi, guys. Looks interesting.

Can I write policies in Pytorch? Or is this tf only?

Thanks.

drozzy commented 4 years ago

Never mind just saw in the paper that the framework is agnostic to DL framework.

aslanides commented 4 years ago

Hi drozzy -- thanks for your interest. Yes! We've tried as much as possible to decouple things from DL frameworks. See e.g. the JAX agents for ideas about how you can do things in PyTorch. Let me know how you get on!

drozzy commented 4 years ago

Thanks @aslanides
Just curious if you looked at erlang for your distribution strategy? A lot of patterns you talk about (e.g. client for polling from learner) come for free out of the box there. For example, lack of distinction between local and remote calls is done via built in rpc module.

(Also see Elixir which is the same bytecode compatible language as erlang)

Good luck.

aslanides commented 4 years ago

Internally at DeepMind we use our own distributed computing stack built on top of gRPC and integrated with Borg and other DeepMind/Google infrastructure. We don't plan to move off this any time soon ;). But thanks for the recommendation, I've heard good things about Elixir.

drozzy commented 4 years ago

I was thinking maybe for public version of acme. :) If and when you decide to add distributed component.

aslanides commented 4 years ago

Yes, understood. What I meant to say is that if/when we do open-source distributed components they'll most likely be simplified versions of what we use internally. You've seen first-hand from the issues with the tutorial colabs that things that we don't use internally are not quite as well maintained as the rest of the codebase ;). So I think our building on a separate stack just for open-source would probably be undesirable for similar reasons.

fastturtle commented 4 years ago

That said, Erlang is certainly an interesting idea for distribution. While it is not what we use internally I think it could be compatible with Acme and would be curious to see how it works out.

AbhayGoyal commented 2 months ago

If I was changing some code which used ACME for jax, how would I convert it to something for pytorch?