List out of Range

[linglingxiansen]

I run this command: python main_llm_zeroshot.py --split val --eval 1 --auto_gpu_config 0 -n 1 --num_eval_episodes 2000 --load pretrained_models/llm_model.pt --use_gtsem 0 --num_local_steps 10

ts/hm3d_v0.2/val/00848-ziup5kvtCCR/.basis.scene_instance.json W1220 05:11:32.560626 922 AttributesManagerBase.h:340] No Glob path result for ./data/scene_datasets/hm3d_v0.2/val/00853-5cdEh9F2hJL/.basis.scene_instance.json W1220 05:11:32.560642 922 AttributesManagerBase.h:340] No Glob path result for ./data/scene_datasets/hm3d_v0.2/val/00873-bxsVRursffK/.basis.scene_instance.json W1220 05:11:32.560659 922 AttributesManagerBase.h:340] No Glob path result for ./data/scene_datasets/hm3d_v0.2/val/00876-mv2HUxq3B53/.basis.scene_instance.json W1220 05:11:32.560676 922 AttributesManagerBase.h:340] No Glob path result for ./data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/.basis.scene_instance.json W1220 05:11:32.560693 922 AttributesManagerBase.h:340] No Glob path result for ./data/scene_datasets/hm3d_v0.2/val/00878-XB4GS9ShBRE/.basis.scene_instance.json W1220 05:11:32.560709 922 AttributesManagerBase.h:340] No Glob path result for ./data/scene_datasets/hm3d_v0.2/val/00880-Nfvxx8J5NCo/.basis.scene_instance.json W1220 05:11:32.560726 922 AttributesManagerBase.h:340] No Glob path result for ./data/scene_datasets/hm3d_v0.2/val/00890-6s7QHgap2fW/.basis.scene_instance.json W1220 05:11:32.560743 922 AttributesManagerBase.h:340] No Glob path result for ./data/scene_datasets/hm3d_v0.2/val/00891-cvZr5TUy5C5/*.basis.scene_instance.json I1220 05:11:32.560751 922 AttributesManagerBase.h:343] ::buildAttrSrcPathsFromJSONAndLoad : 123 paths specified in JSON doc for Scene Instance templates. I1220 05:11:32.560765 922 SceneDatasetAttributesManager.cpp:36] JSON Configuration File (./data/scene_datasets/hm3d_v0.2/hm3d_annotated_basis.scene_dataset_config.json) based dataset attributes created and registered. I1220 05:11:32.560773 922 MetadataMediator.cpp:127] ::createSceneDataset : Dataset ./data/scene_datasets/hm3d_v0.2/hm3d_annotated_basis.scene_dataset_config.json successfully created. I1220 05:11:32.560781 922 MetadataMediator.cpp:230] ::setActiveSceneDatasetName : Attempt to create new dataset ./data/scene_datasets/hm3d_v0.2/hm3d_annotated_basis.scene_dataset_config.json succeeded. Currently active dataset : ./data/scene_datasets/hm3d_v0.2/hm3d_annotated_basis.scene_dataset_config.json I1220 05:11:32.560791 922 MetadataMediator.cpp:183] ::setCurrPhysicsAttributesHandle : Old physics manager attributes changed to ./data/default.physics_config.json successfully. I1220 05:11:32.560801 922 MetadataMediator.cpp:68] ::setSimulatorConfiguration : Set new simulator config for scene/stage : data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb and dataset : ./data/scene_datasets/hm3d_v0.2/hm3d_annotated_basis.scene_dataset_config.json which is currently active dataset. Renderer: NVIDIA GeForce RTX 3090/PCIe/SSE2 by NVIDIA Corporation OpenGL version: 4.6.0 NVIDIA 515.48.07 Using optional features: GL_ARB_vertex_array_object GL_ARB_ES2_compatibility GL_ARB_separate_shader_objects GL_ARB_robustness GL_ARB_texture_storage GL_ARB_invalidate_subdata GL_ARB_texture_storage_multisample GL_ARB_multi_bind GL_ARB_direct_state_access GL_ARB_get_texture_sub_image GL_ARB_texture_filter_anisotropic GL_KHR_debug Using driver workarounds: no-forward-compatible-core-context nv-egl-incorrect-gl11-function-pointers no-layout-qualifiers-on-old-glsl nv-zero-context-profile-mask nv-implementation-color-read-format-dsa-broken nv-cubemap-inconsistent-compressed-image-size nv-cubemap-broken-full-compressed-image-query nv-compressed-block-size-in-bits I1220 05:11:32.658038 922 ManagedFileBasedContainer.h:210] ::convertFilenameToPassedExt : Filename : data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb changed to proposed scene_instance.json filename : data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.scene_instance.json I1220 05:11:32.658088 922 MetadataMediator.cpp:294] ::getSceneAttributesByName : No existing scene instance attributes containing name data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb found in Dataset : ./data/scene_datasets/hm3d_v0.2/hm3d_annotated_basis.scene_dataset_config.json but 1 StageAttributes found. Using ./data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb as stage and to construct a SceneAttributes with same name that will be added to Dataset. I1220 05:11:32.658149 922 SceneDatasetAttributes.cpp:45] ::addNewSceneInstanceToDataset : Dataset : 'hm3d_annotated_basis' : Stage Attributes './data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb' specified in Scene Attributes exists in dataset library. I1220 05:11:32.658156 922 SceneDatasetAttributes.cpp:79] ::addNewSceneInstanceToDataset : Dataset : 'hm3d_annotated_basis' : Lighting Layout Attributes 'no_lights' specified in Scene Attributes but does not exist in dataset, so creating. I1220 05:11:32.658162 922 ManagedFileBasedContainer.h:210] ::convertFilenameToPassedExt : Filename : no_lights changed to proposed lighting_config.json filename : no_lights.lighting_config.json I1220 05:11:32.658172 922 ManagedFileBasedContainer.h:210] ::convertFilenameToPassedExt : Filename : no_lights changed to proposed lighting_config.json filename : no_lights.lighting_config.json I1220 05:11:32.658179 922 AttributesManagerBase.h:365] ::createFromJsonOrDefaultInternal : Proposing JSON name : no_lights.lighting_config.json from original name : no_lights | This file does not exist. I1220 05:11:32.658195 922 LightLayoutAttributesManager.cpp:34] File (no_lights) not found, so new default light layout attributes created and registered. I1220 05:11:32.658203 922 Simulator.cpp:206] ::setSceneInstanceAttributes : Navmesh file location in scene instance : ./data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.navmesh I1220 05:11:32.658211 922 Simulator.cpp:213] ::setSceneInstanceAttributes : Loading navmesh from ./data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.navmesh I1220 05:11:32.658282 922 Simulator.cpp:216] ::setSceneInstanceAttributes : Navmesh Loaded. I1220 05:11:32.658303 922 SceneGraph.h:85] Created DrawableGroup: I1220 05:11:32.658310 922 Simulator.cpp:251] ::setSceneInstanceAttributes : SceneInstance : data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb proposed Semantic Scene Descriptor filename : ./data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.scn E1220 05:11:32.658318 922 SemanticScene.h:155] ::loadSemanticSceneDescriptor : File ./data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.scn does not exist. Aborting load. W1220 05:11:32.658329 922 Simulator.cpp:275] ::setSceneInstanceAttributes : All attempts to load SSD with SceneAttributes-provided name ./data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.scn : exist : 0 : loaded as expected type : 0 I1220 05:11:32.658741 922 Simulator.cpp:324] ::createSceneInstance : Using scene instance-specified Light key : -no_lights- I1220 05:11:32.658753 922 MetadataMediator.cpp:68] ::setSimulatorConfiguration : Set new simulator config for scene/stage : data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb and dataset : ./data/scene_datasets/hm3d_v0.2/hm3d_annotated_basis.scene_dataset_config.json which is currently active dataset. I1220 05:11:32.658794 922 Simulator.cpp:382] ::createSceneInstance : Start to load stage named : ./data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb with render asset : ./data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb and collision asset : ./data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb W1220 05:11:32.658818 922 ResourceManager.cpp:252] ::loadStage : Not loading semantic mesh - File Name : ./data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis_semantic.ply does not exist. I1220 05:11:32.658823 922 ResourceManager.cpp:285] ::loadStage : start load render asset ./data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb. I1220 05:11:32.658826 922 ResourceManager.cpp:699] ::loadStageInternal : Attempting to load stage ./data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb I1220 05:11:32.658838 922 ResourceManager.cpp:1280] Importing Basis files as BC7 for 4ok3usBNeis.basis.glb I1220 05:11:33.523583 922 Simulator.cpp:400] ::createSceneInstance : Successfully loaded stage named : ./data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb W1220 05:11:33.523627 922 Simulator.cpp:435]

Simulator::createSceneInstance : The active scene does not contain semantic annotations.

I1220 05:11:33.523638 922 MetadataMediator.cpp:262] ::getSceneAttributesByName : Query dataset : ./data/scene_datasets/hm3d_v0.2/hm3d_annotated_basis.scene_dataset_config.json for SceneAttributes named : data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb yields 1 candidates. Using data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb. I1220 05:11:33.523686 922 SceneDatasetAttributes.cpp:45] ::addNewSceneInstanceToDataset : Dataset : 'hm3d_annotated_basis' : Stage Attributes './data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb' specified in Scene Attributes exists in dataset library. I1220 05:11:33.523694 922 SceneDatasetAttributes.cpp:85] ::addNewSceneInstanceToDataset : Dataset : 'hm3d_annotated_basis' : Lighting Layout Attributes no_lights specified in Scene Attributes exists in dataset library. I1220 05:11:33.523700 922 MetadataMediator.cpp:262] ::getSceneAttributesByName : Query dataset : ./data/scene_datasets/hm3d_v0.2/hm3d_annotated_basis.scene_dataset_config.json for SceneAttributes named : data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb yields 1 candidates. Using data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb. I1220 05:11:33.523725 922 SceneDatasetAttributes.cpp:45] ::addNewSceneInstanceToDataset : Dataset : 'hm3d_annotated_basis' : Stage Attributes './data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb' specified in Scene Attributes exists in dataset library. I1220 05:11:33.523732 922 SceneDatasetAttributes.cpp:85] ::addNewSceneInstanceToDataset : Dataset : 'hm3d_annotated_basis' : Lighting Layout Attributes no_lights specified in Scene Attributes exists in dataset library. I1220 05:11:33.523752 922 Simulator.cpp:182] Simulator::reconfigure() : createSceneInstance success == true for active scene name : data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.glb with renderer. I1220 05:11:33.524130 922 simulator.py:221] Loaded navmesh data/scene_datasets/hm3d_v0.2/val/00877-4ok3usBNeis/4ok3usBNeis.basis.navmesh I1220 05:11:33.524414 922 simulator.py:233] Recomputing navmesh for agent's height 0.88 and radius 0.18. I1220 05:11:33.535657 922 PathFinder.cpp:382] Building navmesh with 222x162 cells I1220 05:11:33.633204 922 PathFinder.cpp:652] Created navmesh with 283 vertices 136 polygons I1220 05:11:33.633227 922 Simulator.cpp:798] reconstruct navmesh successful 2023-12-20 05:11:33,635 Initializing task ObjectNav-v1 /home/ubuntu/anaconda3/envs/l3mvn/lib/python3.7/site-packages/torchvision/transforms/transforms.py:330: UserWarning: Argument 'interpolation' of type int is deprecated since 0.13 and will be removed in 0.15. Please use InterpolationMode enum. "Argument 'interpolation' of type int is deprecated since 0.13 and will be removed in 0.15. " [12/20 05:11:33 detectron2]: Arguments: Namespace(confidence_threshold=0.9, config_file='configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml', input=['input1.jpeg'], opts=['MODEL.WEIGHTS', 'detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl', 'MODEL.DEVICE', 'cuda:0'], output=None, video_input=None, webcam=False) => loading RedNet checkpoint 'RedNet/model/rednet_semmap_mp3d_40.pth' => loaded checkpoint 'RedNet/model/rednet_semmap_mp3d_40.pth' (epoch 53) Model's state_dict: <habitat_sim._ext.habitat_sim_bindings.SemanticScene object at 0x7f6248164730> [0] [] I1220 05:11:36.921927 922 PhysicsManager.cpp:50] Deconstructing PhysicsManager I1220 05:11:36.921968 922 SceneManager.h:25] Deconstructing SceneManager I1220 05:11:36.921979 922 SceneGraph.h:25] Deconstructing SceneGraph I1220 05:11:36.922381 922 Sensor.cpp:69] Deconstructing Sensor I1220 05:11:36.922648 922 Sensor.cpp:69] Deconstructing Sensor I1220 05:11:36.922828 922 Sensor.cpp:69] Deconstructing Sensor I1220 05:11:36.925798 922 Renderer.cpp:71] Deconstructing Renderer I1220 05:11:36.925817 922 WindowlessContext.h:17] Deconstructing WindowlessContext Process ForkServerProcess-1: Traceback (most recent call last): File "/home/ubuntu/anaconda3/envs/l3mvn/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/home/ubuntu/anaconda3/envs/l3mvn/lib/python3.7/multiprocessing/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/home/ubuntu/L3MVN/envs/habitat/utils/vector_env.py", line 202, in _worker_env observations = env.reset() File "/home/ubuntu/L3MVN/agents/sem_exp.py", line 90, in reset obs, info = super().reset() File "/home/ubuntu/L3MVN/envs/habitat/objectgoal_env21.py", line 151, in reset semantic = self._preprocess_semantic(obs["semantic"]) File "/home/ubuntu/L3MVN/envs/habitat/objectgoal_env21.py", line 221, in _preprocess_semantic if self.scene.objects[se[i]].category.name() in self.hm3d_semantic_mapping: IndexError: list index out of range

and here is my folder: Thanks very much! I really want to implement this project!!!Please help me, thanks~

[ybgdgh]

Hi, you can refer to this issue to fix the problem. Hope this can help you. Thanks

[linglingxiansen]

Hi, you can refer to this issue to fix the problem. Hope this can help you. Thanks @ybgdgh Thanks very much for your help! I have solved that problem. And I have another question, how can I run this project with visualization like the picture below through some modifications in the code? Thank you very much again for your kindly help!

[ybgdgh]

You can add a parameter "-v 1" at the end of your command to show it, and you can also use "--print_images 1" to save the images in your folder.

[linglingxiansen]

Thanks very much again for your kindly help! I have seen the visualization!

[yuhongshi123]

You can add a parameter "-v 1" at the end of your command to show it, and you can also use "--print_images 1" to save the images in your folder.

Hi~ I encountered the following error when using the "-v 1" command to visualize it, it looks like the image is not loaded, how should I modify it? Thanks~

Process ForkServerProcess-1: Traceback (most recent call last): File "/media/syh/anaconda3/envs/al3mvn/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/media/syh/anaconda3/envs/al3mvn/lib/python3.7/multiprocessing/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/media/syh/project1_l3mvn/l3mvn01/L3MVN/envs/habitat/utils/vector_env.py", line 202, in _worker_env observations = env.reset() File "/media/syh/project1_l3mvn/l3mvn01/L3MVN/agents/sem_exp.py", line 114, in reset self.vis_image = vu.init_vis_image(self.goal_name, self.legend) File "/media/syh/project1_l3mvn/l3mvn01/L3MVN/agents/utils/visualization.py", line 62, in init_visimage lx, ly, = legend.shape AttributeError: 'NoneType' object has no attribute 'shape'

[ybgdgh]

you can just ignore lines 114 and 115 since it's the part from another repo.

[yuhongshi123]

you can just ignore lines 114 and 115 since it's the part from another repo.

Thanks for your reply! 1、when i ignore the line 113-114 in sem_exp.py :

if args.visualize or args.print_images:

    #    self.vis_image = vu.init_vis_image(self.goal_name, self.legend)

the error reported：

Process ForkServerProcess-1: Traceback (most recent call last): File "/media/syh/anaconda3/envs/al3mvn/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/media/syh/anaconda3/envs/al3mvn/lib/python3.7/multiprocessing/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/media/syh/project1_l3mvn/l3mvn01/L3MVN/envs/habitat/utils/vector_env.py", line 230, in _worker_env env.plan_act_and_preprocess(data) File "/media/syh/project1_l3mvn/l3mvn01/L3MVN/agents/sem_exp.py", line 167, in plan_act_and_preprocess self._visualize(planner_inputs) File "/media/syh/project1_l3mvn/l3mvn01/L3MVN/agents/sem_exp.py", line 505, in _visualize goal_fmb = skimage.draw.circle_perimeter(f_pos[0][0], f_pos[0][1], local_w/4-2) File "/media/syh/anaconda3/envs/al3mvn/lib/python3.7/site-packages/skimage/draw/draw.py", line 513, in circle_perimeter return _circle_perimeter(r, c, radius, method, shape) File "skimage/draw/_draw.pyx", line 249, in skimage.draw._draw._circle_perimeter TypeError: 'float' object cannot be interpreted as an integer

2、and then i modify line 505：goal_fmb = skimage.draw.circle_perimeter(f_pos[0][0], f_pos[0][1], local_w//4-2)

the error reported：

Process ForkServerProcess-1: Traceback (most recent call last): File "/media/syh/anaconda3/envs/al3mvn/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/media/syh/anaconda3/envs/al3mvn/lib/python3.7/multiprocessing/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/media/syh/project1_l3mvn/l3mvn01/L3MVN/envs/habitat/utils/vector_env.py", line 230, in _worker_env env.plan_act_and_preprocess(data) File "/media/syh/project1_l3mvn/l3mvn01/L3MVN/agents/sem_exp.py", line 167, in plan_act_and_preprocess self._visualize(planner_inputs) File "/media/syh/project1_l3mvn/l3mvn01/L3MVN/agents/sem_exp.py", line 526, in _visualize self.vis_image[50:530, 15:655] = self.rgb_vis TypeError: 'NoneType' object does not support item assignment

Excuse me how should I do next? Thanks*3 !

[ybgdgh]

Sorry, it's my fault. You need to comment out lines 62-63 in L3MVN/agents/utils/visualization.py. I will update it in the repo later. Hope this can help you.

[yuhongshi123]

yeah! it works! the error "TypeError: 'float' object cannot be interpreted as an integer" is reported also, and i modify the line 505：“goal_fmb = skimage.draw.circle_perimeter(f_pos[0][0], f_pos[0][1], int(local_w/4-2))” and then it worked. Thank you very much for your reply and suggestions！