Issue when running hand manipulation tasks

[NoctisZ]

Hi,

Thanks for your amazing work!

I'm new to Isaac Sim, and I'm trying to train/test the ShadowHand and AllegroHand tasks on my Windows machine with the default parameters using the example commands:

PYTHON_PATH scripts/rlgames_train.py task=ShadowHand and PYTHON_PATH scripts/random_policy.py task=ShadowHand

However, I always got the following warnings and nothing shows on Isaac Sim UI:

... Pipeline: GPU Pipeline Device: cuda:0 Sim Device: GPU Obs type: full Task Device: cuda:0 RL device: cuda:0 C:\Research\Omniverse\ov\pkg\isaac_sim-2022.1.1/exts/omni.isaac.ml_archive/pip_prebundle\gym\spaces\box.py:74: UserWarning: WARN: Box bound precision lowered by casting to float32 "Box bound precision lowered by casting to {}".format(self.dtype) [15.269s] [ext: omni.physx.flatcache-1.4.15-5.1] startup 2022-10-13 01:01:26 [15,426ms] [Warning] [carb] Acquiring non optional plugin interface which is not listed as dependency: [omni::kit::IStageUpdate v1.0] (plugin: (null)), by client: omni.physx.flatcache.plugin. Add it to CARB_PLUGIN_IMPL_DEPS() macro of a client. 2022-10-13 01:01:29 [18,563ms] [Warning] [omni.client.plugin] Tick: authentication: Discovery(ws://localhost:3333): Auth/Credentials service not found! 2022-10-13 01:01:41 [30,303ms] [Warning] [omni.hydra.scene_delegate.plugin] Calling getBypassRenderSkelMeshProcessing for prim /World/envs/env_0/shadow_hand/robot0_mfproximal/visuals.proto_robot0_V_mfproximal_id0 that has not been populated 2022-10-13 01:01:42 [31,566ms] [Warning] [omni.physx.plugin] ParseFEMClothMaterial: UsdPrim doesn't have a PhysxSchemaPhysxDeformableSurfaceMaterialAPI

2022-10-13 01:01:44 [32,999ms] [Warning] [omni.usd] Coding Error: in _ValidateEditPrim at line 1279 of C:\b\w\ca6c508eae419cf8\USD\pxr\usd\usd\stage.cpp -- Cannot create prim spec at path </__Master_4/robot0_C_forearm>; authoring to an instancing master is not allowed.

2022-10-13 01:01:44 [33,001ms] [Warning] [omni.usd] Warning: in AddAppliedSchema at line 300 of C:\b\w\ca6c508eae419cf8\USD\pxr\usd\usd\prim.cpp -- Unable to create primSpec at path </__Master_4/robot0_C_forearm> in edit target 'anon:00000189A4CFC0E0:World0.usd'. Failed to add applied API schema.

2022-10-13 01:01:44 [33,003ms] [Warning] [omni.usd] Coding Error: in _ValidateEditPrim at line 1279 of C:\b\w\ca6c508eae419cf8\USD\pxr\usd\usd\stage.cpp -- Cannot create prim spec at path </__Master_4/robot0_C_forearm>; authoring to an instancing master is not allowed.

2022-10-13 01:01:44 [33,003ms] [Warning] [omni.usd] Warning: in AddAppliedSchema at line 300 of C:\b\w\ca6c508eae419cf8\USD\pxr\usd\usd\prim.cpp -- Unable to create primSpec at path </__Master_4/robot0_C_forearm> in edit target 'anon:00000189A4CFC0E0:World0.usd'. Failed to add applied API schema.

... (above warnings repeat)

This problem didn't occur when I run other tasks such as Humanoid and Cartpole.

My system spec: Windows 10 + AMD Ryzen 9 5900HX + RTX3080 laptop

Any help will be greatly appreciated!

[kellyguo11]

Hi @NoctisZ ,

From the commands you've shared, it looks like you are trying to run the process with the viewer. If this is the case, can you try limiting the number of environments? For example, PYTHON_PATH scripts/random_policy.py task=ShadowHand num_envs=10. These warnings should be harmless, but I suspect it's taking too long to load all of the environments in the viewport with 8192 environments. If you are interested in training, I would recommend running in headless mode, PYTHON_PATH scripts/rlgames_train.py task=ShadowHand headless=True, which should reduce the loading time significantly.

[NoctisZ]

You are right, using only 10 environments solved the issue! Thank you so much.