[Help Request] envision_record

High Level Description

I used SMARTS as a outer env, but i failed to visualize in web. So i search the help documents about Data Recording and Replay:

env = gym.make(
    "smarts.env:hiway-v0",
    scenarios=args.scenarios,
    agents={AGENT_ID: agent},
    headless=args.headless,
    envision_record_data_replay_path="./data_replay",
)

i build the env looks like:

        self.base_env = gymnasium.make(
            "smarts.env:hiway-v1",
            sim_name=self.sim_name,
            scenarios=self.scenarios,
            agent_interfaces=self.agent_interface,
            headless=self.headless,
            visdom=self.visdom,
            observation_options=ObservationOptions.multi_agent,
            action_options=ActionOptions.multi_agent,
            seed=self.seed,
            envision_record_data_replay_path="./data_replay",
        )

and get the errors:

  File "/usr/lib/python3.8/site-packages/gymnasium/envs/registration.py", line 652, in make
    raise e
  File "/usr/lib/python3.8/site-packages/gymnasium/envs/registration.py", line 640, in make
    env = env_creator(**_kwargs)
TypeError: __init__() got an unexpected keyword argument 'envision_record_data_replay_path'

i notice that hiway-v1 has an attribution：visualization_client, is this one the 'new' envision_record_data_replay_path? but i still don't know how to use it if so

Version

1.4.0

Operating System

No response

Problems

No response

Hello @knightcalvert, sorry this might not have been documented, for HiWayEnvV1 there is a different way of specifying the argument since Envision is separated from the environment. If you were to set the client explicitly it would be the following way:

visualization_builder = partial(
    Envision,
    endpoint=None, # the IP address if envision is running elsewhere (`None` defaults to `"localhost:8081"`)
    output_dir="./data_replay", # the replacement for `envision_record_data_replay_path`
    headless=False, # A toggle to disable envision
    data_formatter_args=EnvisionDataFormatterArgs("base", enable_reduction=False),
)

env = gym.make(
    "smarts.env:hiway-v1",
    ...,
    visualization_client_builder=visualization_builder,
)

The reason for this is that we did not want to constrain the environment to only use Envision. So any object would work as long as it implements the following minimum interface:

from envision import types as envision_types

class EnvisionInterface:
    def send(state: Optional[envision_types.State]):
        raise NotImplementedError

Hello @knightcalvert, sorry this might not have been documented, for HiWayEnvV1 there is a different way of specifying the argument since Envision is separated from the environment. If you were to set the client explicitly it would be the following way:
visualization_builder = partial(
    Envision,
    endpoint=None, # the IP address if envision is running elsewhere (`None` defaults to `"localhost:8081"`)
    output_dir="./data_replay", # the replacement for `envision_record_data_replay_path`
    headless=False, # A toggle to disable envision
    data_formatter_args=EnvisionDataFormatterArgs("base", enable_reduction=False),
)

env = gym.make(
    "smarts.env:hiway-v1",
    ...,
    visualization_client_builder=visualization_builder,
)
The reason for this is that we did not want to constrain the environment to only use Envision. So any object would work as long as it implements the following minimum interface:
from envision import types as envision_types

class EnvisionInterface:
    def send(state: Optional[envision_types.State]):
        raise NotImplementedError

my intersections.jsonl file already generated, then to replay this file, i use command scl scenario replay -d ./results/envision -t 0.1,

 Traceback (most recent call last):
  File "/home/wang/.local/bin/scl", line 8, in <module>
    sys.exit(scl())
  File "/usr/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "./SMARTS/cli/studio.py", line 105, in replay
    with ThreadPool(len(jsonl_paths)) as pool:
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 922, in __init__
    Pool.__init__(self, processes, initializer, initargs)
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 205, in __init__
    raise ValueError("Number of processes must be at least 1")
ValueError: Number of processes must be at least 1

Hello @knightcalvert, sorry this might not have been documented, for HiWayEnvV1 there is a different way of specifying the argument since Envision is separated from the environment. If you were to set the client explicitly it would be the following way:
visualization_builder = partial(
    Envision,
    endpoint=None, # the IP address if envision is running elsewhere (`None` defaults to `"localhost:8081"`)
    output_dir="./data_replay", # the replacement for `envision_record_data_replay_path`
    headless=False, # A toggle to disable envision
    data_formatter_args=EnvisionDataFormatterArgs("base", enable_reduction=False),
)

env = gym.make(
    "smarts.env:hiway-v1",
    ...,
    visualization_client_builder=visualization_builder,
)
The reason for this is that we did not want to constrain the environment to only use Envision. So any object would work as long as it implements the following minimum interface:
from envision import types as envision_types

class EnvisionInterface:
    def send(state: Optional[envision_types.State]):
        raise NotImplementedError

Hi @Gamenot, I ended up here while going through the issues to find a way to record my envision videos. I tried to use the code snippet you shared, but I could not find the callable 'Envision'. Could you tell me where to import that from or point me to an explanation/documentation of how to record envision videos? Thanks in advance!

huawei-noah / SMARTS