search

ray-project / ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
https://ray.io
Apache License 2.0
34.14k stars 5.8k forks source link

would APE-X and IMPALA work in pure pytorch environment (i.e. without TF installation)? #3877

Closed rouniuyizu closed 5 years ago

rouniuyizu commented 5 years ago

As title. I glanced through the code and seemed we have "import tensorflow..." hardcoded somewhere (like in dqn_policy_graph).

ericl commented 5 years ago

The loss functions are defined in TensorFlow, so yes it is required. Only a couple algorithms are implemented in torch at the moment.

For example, Apex QMIX is written in pytorch.

rouniuyizu commented 5 years ago

The loss functions are defined in TensorFlow, so yes it is required. Only a couple algorithms are implemented in torch at the moment.

For example, Apex QMIX is written in pytorch.

Thanks for the clarification.

It might not be a good idea in mixing TensorFlow based code into Pytorch environment, so any plan in implementing APEX/IMPALA in pytorch? Or mark it in the doc that certain algorithms are only available for TF.

ericl commented 5 years ago

We plan to eventually add PyTorch loss defs for all algorithms, probably PG/PPO next, followed by DQN/APEX. The docs will also be improved once we have a decent set algorithms. Contributions would be welcome of course.

Another interesting direction is to run PyTorch models inside TF graphs as a custom tf op.

On Mon, Jan 28, 2019, 6:08 PM rouniuyizu notifications@github.com wrote:

The loss functions are defined in TensorFlow, so yes it is required. Only a couple algorithms are implemented in torch at the moment.

For example, Apex QMIX is written in pytorch.

Thanks for the clarification.

It might not be a good idea in mixing TensorFlow based code into Pytorch environment, so any plan in implementing APEX/IMPALA in pytorch? Or mark it in the doc where certain algorithms are only available for TF.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ray-project/ray/issues/3877#issuecomment-458379221, or mute the thread https://github.com/notifications/unsubscribe-auth/AAA6SpyKDcUQSYoLftF4rE1-pQ9gKS_Aks5vH605gaJpZM4aVKCs .

rouniuyizu commented 5 years ago

We plan to eventually add PyTorch loss defs for all algorithms, probably PG/PPO next, followed by DQN/APEX. The docs will also be improved once we have a decent set algorithms. Contributions would be welcome of course. Another interesting direction is to run PyTorch models inside TF graphs as a custom tf op. … On Mon, Jan 28, 2019, 6:08 PM rouniuyizu @.***> wrote: The loss functions are defined in TensorFlow, so yes it is required. Only a couple algorithms are implemented in torch at the moment. For example, Apex QMIX is written in pytorch. Thanks for the clarification. It might not be a good idea in mixing TensorFlow based code into Pytorch environment, so any plan in implementing APEX/IMPALA in pytorch? Or mark it in the doc where certain algorithms are only available for TF. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3877 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AAA6SpyKDcUQSYoLftF4rE1-pQ9gKS_Aks5vH605gaJpZM4aVKCs .

Thanks for the update.