Enable proper use of the enableCameraSensors config parameter.
Problem
The current behavior results in enableCameraSensors always being False.
Reason
enable_camera_sensors = config.get("enableCameraSensors", False) always returns False because this is reading from the wrong part of the config. In almost all of the task yaml files, enableCameraSensors is defined under task.env, but in the code it is looking under task. The use of the get and default value made this bug pass by silently, but I noticed it when trying to get my camera sensors to work in a headless training environment.
I also fixed the AllegroHands' yaml files to be consistent with the others (before they were under task.task).
How To Check
Can manually check using:
git grep -B 200 enableCameraSensors:
I also wrote a small python script called check_for_enableCameraSensors.py that does a similar check if placed in IsaacGymEnvs/isaacgymenvs.
# IsaacGymEnvs/isaacgymenvs/check_for_enableCameraSensors.py
from hydra import compose, initialize
from omegaconf import OmegaConf
import isaacgym # Need to import isaacgym before pytorch
from isaacgymenvs.tasks import isaacgym_task_map
from typing import Dict, List, Optional, Any
def find_path_to_keyword(
dictionary: Dict[str, Any], keyword: str, path: Optional[List[str]] = None
) -> List[str]:
"""
EXAMPLE INPUTS:
--------------
data = {
"task": {"env": 1},
"hello": 2
}
keyword = "env"
EXAMPLE OUTPUTS:
--------------
["task", "env"]
"""
if path is None:
path = []
for key, value in dictionary.items():
path.append(key)
if key == keyword:
return path
if isinstance(value, dict):
nested_path = find_path_to_keyword(value, keyword, path)
if nested_path:
return nested_path
path.pop()
return []
initialize(config_path="./cfg")
isaacgym_tasks = list(isaacgym_task_map.keys())
for task in isaacgym_tasks:
# Read task yaml
try:
cfg = compose(config_name="config", overrides=[f"task={task}"])
except:
print(f"Failed to load {task}")
continue
# Get path to enableCameraSensors
path_to_keyword = find_path_to_keyword(
dictionary=OmegaConf.to_container(cfg), keyword="enableCameraSensors"
)
if len(path_to_keyword) == 0:
print(f"enableCameraSensors not in {task}")
continue
print(f"Path to enableCameraSensors in {task}: {path_to_keyword}")
cd IsaacGymEnvs/isaacgymenvs
python check_for_enableCameraSensors.py
Before Change
Note the ['task, 'task', 'enableCameraSensors'] in AllegroHand is inconsistent (it was the same in AllegroHandManualDR and AllegroHandADR too, but failed to load for some reason).
Path to enableCameraSensors in AllegroHand: ['task', 'task', 'enableCameraSensors']
enableCameraSensors not in AllegroKuka
Failed to load AllegroKukaTwoArms
Failed to load AllegroHandManualDR
Failed to load AllegroHandADR
Path to enableCameraSensors in Ant: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in Anymal: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in AnymalTerrain: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in BallBalance: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in Cartpole: ['task', 'env', 'enableCameraSensors']
enableCameraSensors not in FactoryTaskGears
enableCameraSensors not in FactoryTaskInsertion
enableCameraSensors not in FactoryTaskNutBoltPick
enableCameraSensors not in FactoryTaskNutBoltPlace
enableCameraSensors not in FactoryTaskNutBoltScrew
Path to enableCameraSensors in FrankaCabinet: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in FrankaCubeStack: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in Humanoid: ['task', 'env', 'enableCameraSensors']
enableCameraSensors not in HumanoidAMP
Path to enableCameraSensors in Ingenuity: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in Quadcopter: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in ShadowHand: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in Trifinger: ['task', 'env', 'enableCameraSensors']
After
Note the consistent ['task, 'env', 'enableCameraSensors'] now.
Path to enableCameraSensors in AllegroHand: ['task', 'env', 'enableCameraSensors']
enableCameraSensors not in AllegroKuka
Failed to load AllegroKukaTwoArms
Failed to load AllegroHandManualDR
Failed to load AllegroHandADR
Path to enableCameraSensors in Ant: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in Anymal: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in AnymalTerrain: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in BallBalance: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in Cartpole: ['task', 'env', 'enableCameraSensors']
enableCameraSensors not in FactoryTaskGears
enableCameraSensors not in FactoryTaskInsertion
enableCameraSensors not in FactoryTaskNutBoltPick
enableCameraSensors not in FactoryTaskNutBoltPlace
enableCameraSensors not in FactoryTaskNutBoltScrew
Path to enableCameraSensors in FrankaCabinet: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in FrankaCubeStack: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in Humanoid: ['task', 'env', 'enableCameraSensors']
enableCameraSensors not in HumanoidAMP
Path to enableCameraSensors in Ingenuity: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in Quadcopter: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in ShadowHand: ['task', 'env', 'enableCameraSensors']
Path to enableCameraSensors in Trifinger: ['task', 'env', 'enableCameraSensors']
Goal
Enable proper use of the
enableCameraSensors
config parameter.Problem
The current behavior results in
enableCameraSensors
always being False.Reason
enable_camera_sensors = config.get("enableCameraSensors", False)
always returns False because this is reading from the wrong part of the config. In almost all of the task yaml files,enableCameraSensors
is defined undertask.env
, but in the code it is looking undertask
. The use of theget
and default value made this bug pass by silently, but I noticed it when trying to get my camera sensors to work in a headless training environment.I also fixed the AllegroHands' yaml files to be consistent with the others (before they were under
task.task
).How To Check
Can manually check using:
I also wrote a small python script called
check_for_enableCameraSensors.py
that does a similar check if placed inIsaacGymEnvs/isaacgymenvs
.Before Change
Note the ['task, 'task', 'enableCameraSensors'] in AllegroHand is inconsistent (it was the same in AllegroHandManualDR and AllegroHandADR too, but failed to load for some reason).
After
Note the consistent ['task, 'env', 'enableCameraSensors'] now.