search

huangwl18 / ReKep

ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation
https://rekep-robot.github.io/
485 stars 49 forks source link

Isaac-Sim 4.1/4.2版本运行问题 #8

Open YunLongPan opened 3 weeks ago

YunLongPan commented 3 weeks ago

大佬您好，我在运行demo代码 python main.py --use_cached_query 出现以下报错： 我是在Isaac-Sim 4.2和4.1上运行，请问是否因为sim版本的原因呢？

[22.782s] [ext: omni.graph.window.particle.system-105.1.18] startup DYNAMICS_TODO: Make sure "omni.particle.system.core2" is a proper dependency. For now, it can just be set to autoload. DYNAMICS_TODO: Make sure "omni.particle.system.core2" is a proper dependency. For now, it can just be set to autoload. [22.890s] [ext: omni.particle.system.bundle-105.1.0] startup Traceback (most recent call last): File "/home/pci/桌面/ReKep/main.py", line 379, in main = Main(scene_file, visualize=args.visualize) File "/home/pci/桌面/ReKep/main.py", line 40, in init self.env = ReKepOGEnv(global_config['env'], scene_file, verbose=False) File "/home/pci/桌面/ReKep/environment.py", line 69, in init self.og_env = og.Environment(dict(scene=self.config['scene'], robots=[self.config['robot']['robot_config']], env=self.config['og_sim'])) File "/home/pci/桌面/OmniGibson/omnigibson/utils/python_utils.py", line 93, in wrapper func(*values.args, *values.kwargs) File "/home/pci/桌面/OmniGibson/omnigibson/envs/env_base.py", line 103, in init og.launch( File "/home/pci/桌面/OmniGibson/omnigibson/simulator.py", line 1780, in _launch_simulator Simulator(args, **kwargs) File "/home/pci/桌面/OmniGibson/omnigibson/simulator.py", line 295, in init self._validate_dts(physics_dt, rendering_dt, sim_step_dt) File "/home/pci/桌面/OmniGibson/omnigibson/simulator.py", line 508, in _validate_dts assert math.isclose( AssertionError: Rendering dt (0.03333333333333333) must be a multiple of physics dt (0.01) 2024-10-06 11:12:02 [22,877ms] [Warning] [carb] [Plugin: omni.sensors.tiled.plugin] Module /home/pci/software/anaconda3/envs/rpk/lib/python3.10/site-packages/isaacsim/extscache/omni.sensors.tiled-0.0.4+106.0.0.lx64.r/bin/libomni.sensors.tiled.plugin.so remained loaded after unload request 2024-10-06 11:12:02 [22,890ms] [Warning] [carb] [Plugin: omni.spectree.delegate.plugin] Module /home/pci/software/anaconda3/envs/rpk/lib/python3.10/site-packages/isaacsim/extscache/omni.usd_resolver/bin/libomni.spectree.delegate.plugin.so remained loaded after unload request 2024-10-06 11:12:02 [22,898ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-06 11:12:02 [22,898ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-06 11:12:02 [22,898ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-06 11:12:02 [22,898ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-06 11:12:02 [22,898ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-06 11:12:02 [22,898ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-06 11:12:02 [22,898ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-06 11:12:02 [22,898ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-06 11:12:02 [22,898ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-06 11:12:02 [22,898ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-06 11:12:02 [22,898ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-06 11:12:02 [22,898ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Core' for removal 2024-10-06 11:12:02 [22,905ms] [Warning] [omni.physx.plugin] Detach stage failed! 2024-10-06 11:12:02 [23,036ms] [Warning] [carb] Recursive unloadAllPlugins() detected!

salemon commented 3 weeks ago

I had the same issue

YunLongPan commented 3 weeks ago

使用的是Isaac-Sim 4.1版本，目前问题已解决。可能是一些安装包更新后，导致源码中一系列的参数设置、运算和数据类型不一致等问题，从报错部分逐步debug即可解决。 例如：File "/home/pci/桌面/OmniGibson/omnigibson/simulator.py", line 508, in _validate_dts assert math.isclose( AssertionError: Rendering dt (0.03333333333333333) must be a multiple of physics dt (0.01)

Gutilence14 commented 3 weeks ago

我使用的是Isaac-Sim 4.1，遇到同样的问题，请问大佬修改了哪些内容，报错如下 2024-10-09 05:37:32 [134,486ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead. [134.572s] [ext: omni.kit.widget.zoombar-1.0.5] startup [134.578s] [ext: omni.scene.visualization.core-105.4.13] startup 2024-10-09 05:37:32 [134,523ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead. [134.594s] [ext: omni.ramp-105.1.15] startup [134.603s] [ext: omni.kit.browser.core-2.3.11] startup [134.611s] [ext: omni.particle.system.core-105.1.6] startup 2024-10-09 05:37:32 [134,563ms] [Warning] [omni.particle.system.core.scripts.extension] ATTENTION!: omni.particle.system.core is currently undergoing extensive (breaking) changes. Please be aware that systems built with the existing extension will need to be rebuilt in the USD Composer 2023.2 release. [134.638s] [ext: omni.kit.browser.folder.core-1.9.12] startup [134.648s] [ext: omni.particle.system.ui-105.1.10] startup [134.907s] [ext: omni.graph.window.particle.system-105.1.18] startup DYNAMICS_TODO: Make sure "omni.particle.system.core2" is a proper dependency. For now, it can just be set to autoload. DYNAMICS_TODO: Make sure "omni.particle.system.core2" is a proper dependency. For now, it can just be set to autoload. [134.984s] [ext: omni.particle.system.bundle-105.1.0] startup [134.999s] [ext: omni.kit.window.viewport-0.0.0] startup 2024-10-09 05:37:32 [135,072ms] [Warning] [omni.hydra] GPU memory budget disabled, memory manager RendererMemoryBudgetGPU already exists. Traceback (most recent call last): File "D:\AI\Project\ReKep\main.py", line 379, in main = Main(scene_file, visualize=args.visualize) File "D:\AI\Project\ReKep\main.py", line 40, in init self.env = ReKepOGEnv(global_config['env'], scene_file, verbose=False) File "D:\AI\Project\ReKep\environment.py", line 69, in init self.og_env = og.Environment(dict(scene=self.config['scene'], robots=[self.config['robot']['robot_config']], env=self.config['og_sim'])) File "D:\AI\Project\OmniGibson\omnigibson\utils\python_utils.py", line 93, in wrapper func(*values.args, *values.kwargs) File "D:\AI\Project\OmniGibson\omnigibson\envs\env_base.py", line 103, in init og.launch( File "D:\AI\Project\OmniGibson\omnigibson\simulator.py", line 1780, in _launch_simulator Simulator(args, **kwargs) File "D:\AI\Project\OmniGibson\omnigibson\simulator.py", line 295, in init self._validate_dts(physics_dt, rendering_dt, sim_step_dt) File "D:\AI\Project\OmniGibson\omnigibson\simulator.py", line 508, in _validate_dts assert math.isclose( AssertionError: Rendering dt (0.03333333333333333) must be a multiple of physics dt (0.01) 2024-10-09 05:37:32 [135,164ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-09 05:37:32 [135,164ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-09 05:37:32 [135,165ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-09 05:37:32 [135,165ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-09 05:37:32 [135,165ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-09 05:37:32 [135,165ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-09 05:37:32 [135,165ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-09 05:37:32 [135,166ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-09 05:37:32 [135,166ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-09 05:37:32 [135,166ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-09 05:37:32 [135,166ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal 2024-10-09 05:37:32 [135,166ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Core' for removal 2024-10-09 05:37:33 [135,168ms] [Warning] [omni.physx.plugin] Detach stage failed! 2024-10-09 05:37:33 [135,233ms] [Warning] [carb] Recursive unloadAllPlugins() detected!

tkdggg commented 3 weeks ago

trouble2

Bailey-24 commented 3 weeks ago

把config.yaml的physics_frequency参数改为30，或者60就行

Bailey-24 commented 3 weeks ago 
[INFO] [omnigibson.simulator] ---------- Welcome to OmniGibson! ----------
Traceback (most recent call last):
  File "/omnigibson-src/ReKep/main.py", line 379, in <module>
    main = Main(scene_file, visualize=args.visualize)
  File "/omnigibson-src/ReKep/main.py", line 40, in __init__
    self.env = ReKepOGEnv(global_config['env'], scene_file, verbose=False)
  File "/omnigibson-src/ReKep/environment.py", line 69, in __init__
    self.og_env = og.Environment(dict(scene=self.config['scene'], robots=[self.config['robot']['robot_config']], env=self.config['og_sim']))
  File "/omnigibson-src/omnigibson/utils/python_utils.py", line 93, in wrapper
    func(*values.args, **values.kwargs)
  File "/omnigibson-src/omnigibson/envs/env_base.py", line 128, in __init__
    self.load()
  File "/omnigibson-src/omnigibson/envs/env_base.py", line 429, in load
    self._load_scene()
  File "/omnigibson-src/omnigibson/envs/env_base.py", line 257, in _load_scene
    og.sim.import_scene(self._scene)
  File "/omnigibson-src/omnigibson/simulator.py", line 700, in import_scene
    self._last_scene_edge = scene.load(
  File "/omnigibson-src/omnigibson/scenes/scene_base.py", line 449, in load
    new_scene_edge = self._load_scene_prim_with_objects(**kwargs)
  File "/omnigibson-src/omnigibson/scenes/scene_base.py", line 364, in _load_scene_prim_with_objects
    self.add_object(obj)
  File "/omnigibson-src/omnigibson/scenes/scene_base.py", line 630, in add_object
    prim = obj.load(self)
  File "/omnigibson-src/omnigibson/objects/object_base.py", line 123, in load
    prim = super().load(scene)
  File "/omnigibson-src/omnigibson/prims/prim_base.py", line 116, in load
    self._post_load()
  File "/omnigibson-src/omnigibson/objects/dataset_object.py", line 234, in _post_load
    assert th.all(scale > 1e-4), f"Scale of {self.name} is too small: {scale}"
TypeError: all() received an invalid combination of arguments - got (bool), but expected one of:
 * (Tensor input, *, Tensor out)
 * (Tensor input, tuple of ints dim, bool keepdim, *, Tensor out)
 * (Tensor input, int dim, bool keepdim, *, Tensor out)
 * (Tensor input, name dim, bool keepdim, *, Tensor out)

你们会遇到这个问题吗，怎么解决

Gutilence14 commented 3 weeks ago 
[INFO] [omnigibson.simulator] ---------- Welcome to OmniGibson! ----------
Traceback (most recent call last):
  File "/omnigibson-src/ReKep/main.py", line 379, in <module>
    main = Main(scene_file, visualize=args.visualize)
  File "/omnigibson-src/ReKep/main.py", line 40, in __init__
    self.env = ReKepOGEnv(global_config['env'], scene_file, verbose=False)
  File "/omnigibson-src/ReKep/environment.py", line 69, in __init__
    self.og_env = og.Environment(dict(scene=self.config['scene'], robots=[self.config['robot']['robot_config']], env=self.config['og_sim']))
  File "/omnigibson-src/omnigibson/utils/python_utils.py", line 93, in wrapper
    func(*values.args, **values.kwargs)
  File "/omnigibson-src/omnigibson/envs/env_base.py", line 128, in __init__
    self.load()
  File "/omnigibson-src/omnigibson/envs/env_base.py", line 429, in load
    self._load_scene()
  File "/omnigibson-src/omnigibson/envs/env_base.py", line 257, in _load_scene
    og.sim.import_scene(self._scene)
  File "/omnigibson-src/omnigibson/simulator.py", line 700, in import_scene
    self._last_scene_edge = scene.load(
  File "/omnigibson-src/omnigibson/scenes/scene_base.py", line 449, in load
    new_scene_edge = self._load_scene_prim_with_objects(**kwargs)
  File "/omnigibson-src/omnigibson/scenes/scene_base.py", line 364, in _load_scene_prim_with_objects
    self.add_object(obj)
  File "/omnigibson-src/omnigibson/scenes/scene_base.py", line 630, in add_object
    prim = obj.load(self)
  File "/omnigibson-src/omnigibson/objects/object_base.py", line 123, in load
    prim = super().load(scene)
  File "/omnigibson-src/omnigibson/prims/prim_base.py", line 116, in load
    self._post_load()
  File "/omnigibson-src/omnigibson/objects/dataset_object.py", line 234, in _post_load
    assert th.all(scale > 1e-4), f"Scale of {self.name} is too small: {scale}"
TypeError: all() received an invalid combination of arguments - got (bool), but expected one of:
 * (Tensor input, *, Tensor out)
 * (Tensor input, tuple of ints dim, bool keepdim, *, Tensor out)
 * (Tensor input, int dim, bool keepdim, *, Tensor out)
 * (Tensor input, name dim, bool keepdim, *, Tensor out)

你们会遇到这个问题吗，怎么解决

og_scene_file_pen.json第897行中 "scale": 2 改成 "scale": [ 2.0, 2.0, 2.0 ],

Bailey-24 commented 2 weeks ago

哈喽，我运行python main.py --use_cached_query， 但一开始笔会直接掉到桌子下面，你们有遇到吗，怎么解决比较好。 image