Ray RL | Local Env Runner spinning up when it shouldn't

[H-Park]

What happened + What you expected to happen

When setting algo.create_env_on_local_worker = False, a local env runner still gets created due to ray hard coding it to do so.

As a user, I do not expect double the amount of sims to be spun up than what I specify.

Versions / Dependencies

Ray: 2.30

Reproduction script

PPOConfig()
            .api_stack(
                enable_rl_module_and_learner=True,
                enable_env_runner_and_connector_v2=True
            )
            .env_runners(
                create_env_on_local_worker=False
            )

Issue Severity

Medium: It is a significant difficulty but I can work around it.

[simonsays1980]

@H-Park The local EnvRunner is always existent as it is the dirvers hub to sync weights with remote learners and remote evaluation EnvRunners. After the summit we will take closer looks at external environments and server-client modes. During this iteration we will also check, if we could provide the local EnvRunner without environment.

Is there a specific use case in which the environment on the local EnvRunner is blocking your work?

[H-Park]

The main issue is this produces a major bloat on resources for large sims.

[XavierGeerinck]

Hi @simonsays1980, any update on it, or workaround? We are seeing that this is causing quite some frustrations. With 8 envs on an env runner we suddenly see 16 spinning up due to the default local one