cremebrule / digital-cousins

Codebase for Automated Creation of Digital Cousins for Robust Policy Learning
https://digital-cousins.github.io
Apache License 2.0
137 stars 15 forks source link

gpt #3

Closed Nimingez closed 1 month ago

Nimingez commented 1 month ago

Querying GPT-4o API... Traceback (most recent call last): File "/home/nmz/anaconda3/envs/acdc/lib/python3.10/site-packages/requests/models.py", line 974, in json return complexjson.loads(self.text, **kwargs) File "/home/nmz/anaconda3/envs/acdc/lib/python3.10/json/init.py", line 346, in loads return _default_decoder.decode(s) File "/home/nmz/anaconda3/envs/acdc/lib/python3.10/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/home/nmz/anaconda3/envs/acdc/lib/python3.10/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/nmz/mgit/digital-cousins/digital_cousins/pipeline/acdc.py", line 202, in main(args) File "/home/nmz/mgit/digital-cousins/digital_cousins/pipeline/acdc.py", line 170, in main pipeline.run( File "/home/nmz/mgit/digital-cousins/digital_cousins/pipeline/acdc.py", line 114, in run success, step_1_output_path = step_1( File "/home/nmz/mgit/digital-cousins/digital_cousins/pipeline/extraction.py", line 180, in call gpt_text_response = gpt(payload=obj_caption_prompt_payload, verbose=self.verbose) File "/home/nmz/mgit/digital-cousins/digital_cousins/models/gpt.py", line 45, in call if "choices" not in response.json().keys(): File "/home/nmz/anaconda3/envs/acdc/lib/python3.10/site-packages/requests/models.py", line 978, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Is this caused by GPT? <Response [200]>

RogerDAI1217 commented 1 month ago

Hi, thanks for checking out our repo. Yes, this is a GPT error. As described in User Tips and Limitations 7, if you meet this error in Step 2 of our pipeline, you can set FeatureMatcher.pipeline.DigitalCousinMatcher.start_at_name to start at the object causing this error.

Nimingez commented 1 month ago

AttributeError: module 'faiss' has no attribute 'StandardGpuResources' pip list my faiss is 1.8.0

cremebrule commented 1 month ago

How did you install faiss? There's a cpu and gpu version -- you need to make sure to install the gpu one (installed via conda, see the README).

Nimingez commented 1 month ago

install one-liner. Now I am trying step-by-step

Nimingez commented 1 month ago

[INFO] [omnigibson.simulator] ---------- Welcome to OmniGibson! ---------- ############################## [Scene 1 / 3] Traceback (most recent call last): File "/home/nmz/anaconda3/envs/acdc-ma/lib/python3.10/site-packages/isaacsim/extscache/omni.kit.widget.viewport/omni/kit/widget/viewport/impl/texture.py", line 355, in on_drawable_changed self.drawable_changed_fn(texture['rp_resource'], event.payload.get('presentation_key', 0)) File "/home/nmz/anaconda3/envs/acdc-ma/lib/python3.10/site-packages/isaacsim/extscache/omni.kit.widget.viewport/omni/kit/widget/viewport/widget.py", line 464, in set_image_data vp_api._sync_viewport_api(camera, canvas_size, vp_api.time, view, projection, force_update=True) File "/home/nmz/anaconda3/envs/acdc-ma/lib/python3.10/site-packages/isaacsim/extscache/omni.kit.widget.viewport/omni/kit/widget/viewport/api.py", line 544, in _sync_viewport_api self.projection = self._conform_projection(policy, camera, image_aspect, canvas_aspect, projection) File "/home/nmz/anaconda3/envs/acdc-ma/lib/python3.10/site-packages/isaacsim/extscache/omni.kit.widget.viewport/omni/kit/widget/viewport/api.py", line 473, in _conform_projection gf_camera = camera.GetCamera(self.__time) AttributeError: 'NoneType' object has no attribute 'GetCamera' Traceback (most recent call last): File "/home/nmz/anaconda3/envs/acdc-ma/lib/python3.10/site-packages/isaacsim/extsPhysics/omni.physx.ui/omni/physxui/scripts/extension.py", line 82, in on_stage_update cam_pos = camera.position_world File "/home/nmz/anaconda3/envs/acdc-ma/lib/python3.10/site-packages/isaacsim/extscache/omni.kit.viewport.utility/omni/kit/viewport/utility/camera_state.py", line 100, in position_world return self.usd_camera.ComputeLocalToWorldTransform(self.time).Transform(Gf.Vec3d(0, 0, 0)) File "/home/nmz/anaconda3/envs/acdc-ma/lib/python3.10/site-packages/isaacsim/extscache/omni.kit.viewport.utility/omni/kit/viewport/utility/camera_state.py", line 87, in usd_camera raise RuntimeError(f'"{self.__camera_path}" is not a valid Usd.Prim or UsdGeom.Camera') RuntimeError: "/World/viewer_camera" is not a valid Usd.Prim or UsdGeom.Camera Isaac error!

RogerDAI1217 commented 1 month ago

Hi, does this actually terminate the process? When we run our pipeline, this also appears. While there are some annoying warnings from the simulator, they don't seem to affect the final result.

cremebrule commented 1 month ago

The specific Isaac error you mention doesn't actually impact runtime. It's a bit of an annoying quirk unfortunately -- we're looking into seeing if we can suppress that one.

As long as you get the OmniGibson shut down successfully! message at the end, then the process has terminated successfully!

cremebrule commented 1 month ago

Hmmm the error seems to suggest that the OmniGibson dataset wasn't installed properly. Do you have assets and og_dataset directories located under omnigibson/data?