[runtime_env] Actors always depend global `pip` field for `runtime_env`

[shrekris-anyscale]

What happened + What you expected to happen

I tried to launch an actor that had no runtime_env dependencies, but it didn't get started until the top level runtime_env passed into ray.init() finished installing.

I expect to be able to either:

1. install the runtime_env fully before the actor runs, so the actor doesn't need to wait for the installation.
2. specify a per-actor runtime_env that fully overrides the parent runtime_env, so the actor doesn't need to wait for the installation.

Versions / Dependencies

Ray on the latest master.

Reproduction script

# repro.py

import ray

runtime_env = {"pip": ["this_is_not_a_real_package_asadf"]}
ray.init(runtime_env=runtime_env, namespace="serve")

@ray.remote(
    runtime_env={
        "pip": [],
    }
)
class A:

    def __call__(self, *args, **kwargs):
        return "hi"

print("check2")

a = A.remote()

print("check3")

print(ray.get(a.__call__.remote()))

print("check4")

$ python repro.py

2023-03-22 16:48:16,536 INFO worker.py:1550 -- Started a local Ray instance. View the dashboard at http://127.0.0.1:8265 
check2
check3
Traceback (most recent call last):
  File "repro.py", line 22, in <module>
    print(ray.get(a.__call__.remote()))
  File "/Users/shrekris/Desktop/ray/python/ray/_private/client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "/Users/shrekris/Desktop/ray/python/ray/_private/worker.py", line 2428, in get
    raise value
ray.exceptions.RuntimeEnvSetupError: Failed to set up runtime environment.
Could not create the actor because its associated runtime env failed to be created.
Traceback (most recent call last):
  File "/Users/shrekris/Desktop/ray/python/ray/dashboard/modules/runtime_env/runtime_env_agent.py", line 357, in _create_runtime_env_with_retry
    runtime_env_context = await asyncio.wait_for(
  File "/Users/shrekris/miniforge3/envs/ae/lib/python3.8/asyncio/tasks.py", line 494, in wait_for
    return fut.result()
  File "/Users/shrekris/Desktop/ray/python/ray/dashboard/modules/runtime_env/runtime_env_agent.py", line 312, in _setup_runtime_env
    await create_for_plugin_if_needed(
  File "/Users/shrekris/Desktop/ray/python/ray/_private/runtime_env/plugin.py", line 252, in create_for_plugin_if_needed
    size_bytes = await plugin.create(uri, runtime_env, context, logger=logger)
  File "/Users/shrekris/Desktop/ray/python/ray/_private/runtime_env/pip.py", line 473, in create
    return await task
  File "/Users/shrekris/Desktop/ray/python/ray/_private/runtime_env/pip.py", line 455, in _create_for_hash
    await PipProcessor(
  File "/Users/shrekris/Desktop/ray/python/ray/_private/runtime_env/pip.py", line 361, in _run
    await self._install_pip_packages(
  File "/Users/shrekris/Desktop/ray/python/ray/_private/runtime_env/pip.py", line 337, in _install_pip_packages
    await check_output_cmd(pip_install_cmd, logger=logger, cwd=cwd, env=pip_env)
  File "/Users/shrekris/Desktop/ray/python/ray/_private/runtime_env/utils.py", line 101, in check_output_cmd
    raise SubprocessCalledProcessError(
ray._private.runtime_env.utils.SubprocessCalledProcessError: Run cmd[9] failed with the following details.
Command '['/tmp/ray/session_2023-03-22_16-48-09_709570_42993/runtime_resources/pip/8dfa33a97a688ee7b784034df121eaf4ff642542/virtualenv/bin/python', '-m', 'pip', 'install', '--disable-pip-version-check', '--no-cache-dir', '-r', '/tmp/ray/session_2023-03-22_16-48-09_709570_42993/runtime_resources/pip/8dfa33a97a688ee7b784034df121eaf4ff642542/requirements.txt']' returned non-zero exit status 1.
Last 50 lines of stdout:
    ERROR: Could not find a version that satisfies the requirement this_is_not_a_real_package_asadf (from versions: none)
    ERROR: No matching distribution found for this_is_not_a_real_package_asadf

Issue Severity

High: It blocks me from completing my task.

[rkooo567]

Seems to be a API correctness bug