ray-project / ray

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
https://ray.io
Apache License 2.0
33.34k stars 5.64k forks source link

[RLlib][A3C] `use_critic` being incorrectly assigned to `lr_schedule` #35579

Closed Fassty closed 1 year ago

Fassty commented 1 year ago

What happened + What you expected to happen

The value for use_critic is incorrectly assigned to lr_schedule in a3c.py. This silently passes for use_critic=False with the value for self.use_critic not being set correctly and raises an error for use_critic=True.

E    lr_schedule, outside_value=lr_schedule[-1][-1], framework=None
E  TypeError: 'bool' object is not subscriptable

Versions / Dependencies

Present in master branch

Reproduction script

import ray
from ray.rllib.algorithms.a3c import A3CConfig

ray.init()

a3c = (
    A3CConfig()
    .environment('CartPole-v1')
    .training(use_critic=True)
    .build()
)

Issue Severity

High: It blocks me from completing my task.

Rohan138 commented 1 year ago

Hi @Fassty could you reopen your PR, but move your changes to the A3C folder inside ray/rllib_contrib instead? Thank you!

Edit: NVM, fixed