search

Learning-and-Intelligent-Systems / kitchen-worlds

A library of long-horizon Task-and-Motion-Planning (TAMP) problems in kitchen and household scenes, as well as planners to solve them
https://learning-and-intelligent-systems.github.io/kitchen-worlds/
MIT License
67 stars 11 forks source link

Report some bugs #11

Closed CeHao-NUS closed 1 week ago

CeHao-NUS commented 1 week ago

1. python examples/test_data_generation.py --config_name kitchen_full_pr2.yaml

 File "/home/crslab/xx/kitchen-worlds/pddlstream/pddlstream/algorithms/diverse.py", line 82, in diverse_from_task
    plan, cost = run_search(temp_dir, debug=debug, planner=planner,
TypeError: run_search() got an unexpected keyword argument 'unique_optimistic'

I manually fix this. But it stopped.

image


Setting negate=True for stream [test-cfree-pose-pose]
Setting negate=True for stream [test-cfree-approach-pose]
Streams: [sample-pose:('?o', '?r')->('?p',), sample-pose-inside:('?o', '?r')->('?p',), sample-grasp:('?o',)->('?g',), inverse-reachability:('?a', '?o', '?p', '?g')->('?q',), inverse-kinematics:('?a', '?o', '?p', '?g', '?q')->('?t',), plan-base-motion:('?q1', '?q2')->('?t',), get-joint-position-open:('?o', '?p1')->('?p2',), sample-handle-grasp:('?o',)->('?g',), inverse-kinematics-grasp-handle:('?a', '?o', '?p', '?g')->('?q', '?aq', '?t'), inverse-kinematics-ungrasp-handle:('?a', '?o', '?p', '?g', '?q', '?aq1')->('?aq2', '?t'), plan-base-pull-handle:('?a', '?o', '?p1', '?p2', '?g', '?q1', '?aq')->('?q2', '?bt')]
Functions: []
Negated: [test-cfree-pose-pose:('?o1', '?p1', '?o2', '?p2')->(), test-cfree-approach-pose:('?o1', '?p1', '?g1', '?o2', '?p2')->()]
Optimizers: []

Iteration: 1 | Complexity: 5 | Skeletons: 0 | Skeleton Queue: 0 | Disabled: 0 | Evaluations: 23 | Eager Calls: 0 | Cost: inf | Search Time: 0.000 | Sample Time: 0.000 | Total Time: 0.000
Attempt: 1 | Results: 14 | Depth: 1 | Success: False | Time: 0.040
Attempt: 2 | Results: 14 | Depth: 0 | Success: True | Time: 0.078
Stream plan (5, 5, 12.001):
   1 > sample-handle-grasp:((15, 1))->(#g0)
   2 > get-joint-position-open:((15, 1), pstn0=0.0)->(#p0)
   3 > inverse-kinematics-grasp-handle:(left, (15, 1), pstn0=0.0, #g0)->(#q0, #a0, #t10)
   4 > plan-base-pull-handle:(left, (15, 1), pstn0=0.0, #p0, #g0, #q0, #a0)->(#q1, #b0)
   5 > plan-base-motion:(q848=(2.0, 3.0, 0.2, 3.142), #q0)->(#t19)
Action plan (3, 2.000):
   move_base(q848=(2.0, 3.0, 0.2, 3.142), #q0, #t19)
   grasp_handle(left, (15, 1), pstn0=0.0, #g0, #q0, aq992=(0.677, -0.343, 1.2, -1.467, 1.242, -1.954, 2.223), #a0, #t10)
   pull_handle(left, (15, 1), pstn0=0.0, #p0, #g0, #q0, #q1, #b0, #a0)

   iter=0, outs=1) sample-handle-grasp:((15, 1))->[(hg496=(-0.376, -0.489, 0.212, -3.142, -0.0, 0.0))]
        sample_joint_position_gen((15, 1), 0.0, closed=False, p_max=1.8) choosing from [1.601, 1.723, 1.657, 1.76, 1.576, 1.722], joint limits = [0.0, 1.8]
   iter=0, outs=1) get-joint-position-open:((15, 1), pstn0=0.0)->[(pstn3=1.723)]
Loaded: /home/crslab/cehao/kitchen-worlds/pybullet_planning/databases/side_left_ir.pickle (0.004 sec)
mobile_streams.get_ik_gen |  exceeding ir_max_attempts = 80
   iter=inf, outs=0) inverse-kinematics-grasp-handle:(left, (15, 1), pstn0=0.0, hg496=(-0.376, -0.489, 0.212, -3.142, -0.0, 0.0))->[]
   iter=1, outs=1) sample-handle-grasp:((15, 1))->[(hg544=(-0.376, -0.559, 0.212, -3.142, -0.0, 0.0))]
mobile_streams.get_ik_gen |  exceeding ir_max_attempts = 80
   iter=inf, outs=0) inverse-kinematics-grasp-handle:(left, (15, 1), pstn0=0.0, hg544=(-0.376, -0.559, 0.212, -3.142, -0.0, 0.0))->[]
   iter=2, outs=1) sample-handle-grasp:((15, 1))->[(hg928=(-0.376, -0.419, 0.212, -3.142, -0.0, 0.0))]
mobile_streams.get_ik_gen |  exceeding ir_max_attempts = 80
   iter=inf, outs=0) inverse-kinematics-grasp-handle:(left, (15, 1), pstn0=0.0, hg928=(-0.376, -0.419, 0.212, -3.142, -0.0, 0.0))->[]
   iter=3, outs=1) sample-handle-grasp:((15, 1))->[(hg248=(-0.376, -0.489, 0.212, -3.142, -0.0, -3.142))]
mobile_streams.get_ik_gen |  exceeding ir_max_attempts = 80
   iter=inf, outs=0) inverse-kinematics-grasp-handle:(left, (15, 1), pstn0=0.0, hg248=(-0.376, -0.489, 0.212, -3.142, -0.0, -3.142))->[]
   iter=4, outs=1) sample-handle-grasp:((15, 1))->[(hg976=(-0.376, -0.419, 0.212, -3.142, -0.0, -3.142))]
mobile_streams.get_ik_gen |  exceeding ir_max_attempts = 80
   iter=inf, outs=0) inverse-kinematics-grasp-handle:(left, (15, 1), pstn0=0.0, hg976=(-0.376, -0.419, 0.212, -3.142, -0.0, -3.142))->[]
   iter=5, outs=1) sample-handle-grasp:((15, 1))->[(hg72=(-0.376, -0.559, 0.212, -3.142, -0.0, -3.142))]
mobile_streams.get_ik_gen |  exceeding ir_max_attempts = 80
   iter=inf, outs=0) inverse-kinematics-grasp-handle:(left, (15, 1), pstn0=0.0, hg72=(-0.376, -0.559, 0.212, -3.142, -0.0, -3.142))->[]
   iter=inf, outs=0) sample-handle-grasp:((15, 1))->[]

2. python examples/test_data_generation.py --config_name kitchen_full_feg.yaml

The scene just freezer.


Solved: True
Cost: 2.000
Length: 2
Deferred: 0
Evaluations: 2
 1) move_cartesian q88=(2.0, 3.0, 0.7, 0.0, -1.57, 0.0) q272=(0.869, 3.801, 1.181, -4.715, -1.061, 7.851) c416=t(6, 53)
 2) pick_hand hand 31 p0=(0.701, 3.707, 1.102, 0.0, -0.0, 0.509) g16=(0.149, 0.0, 0.078, 0.0, -1.571, 0.0) q272=(0.869, 3.801, 1.181, -4.715, -1.061, 7.851) c632=t(6, 2)
[world.object] cannot find object 35

written /home/crslab/xx/kitchen-worlds/outputs/test_feg_kitchen_full/241105_213833/scene.lisdf

[world.object] cannot find object 35

stream_agent.post_process ...

SAVED DATA in /home/crslab/xx/kitchen-worlds/outputs/test_feg_kitchen_full/241105_213833

etUrdfImporter.cpp,126]:

b3Printf: No inertial data for link, using mass=1, localinertiadiagonal = 1,1,1, identity local inertial frame
b3Printf: b3Warning[examples/Importers/ImportURDFDemo/BulletUrdfImporter.cpp,126]:

b3Printf: link_1
b3Printf: b3Warning[examples/Importers/ImportURDFDemo/BulletUrdfImporter.cpp,126]:

b3Printf: No inertial data for link, using mass=1, localinertiadiagonal = 1,1,1, identity local inertial frame
b3Printf: b3Warning[examples/Importers/ImportURDFDemo/BulletUrdfImporter.cpp,126]:

b3Printf: link_1_helper
Control?
0 t(6, 53)

python your_project_folder/run_generation_pigi_custom.py

Failed. 


b3Printf: link_0_helper
Traceback (most recent call last):
  File "your_project_folder/run_generation_pigi_custom.py", line 10, in <module>
    run_agent(
  File "/home/crslab/cehao/kitchen-worlds/pybullet_planning/cogarch_tools/cogarch_run.py", line 83, in run_agent
    state, exogenous, goals, problem_dict = get_pddlstream_problem(args, world_builder_args=world_builder_args,
  File "/home/crslab/cehao/kitchen-worlds/pybullet_planning/cogarch_tools/cogarch_utils.py", line 332, in get_pddlstream_problem
    return problem_fn(args, **kwargs, **problem_kwargs)
  File "/home/crslab/cehao/kitchen-worlds/pybullet_planning/problem_sets/pr2_problems_pigi.py", line 61, in test_full_kitchen
    return test_full_kitchen_domain(args, loader_fn, **kwargs)
  File "/home/crslab/cehao/kitchen-worlds/pybullet_planning/problem_sets/pr2_problems_pigi.py", line 16, in test_full_kitchen_domain
    return problem_template(args, robot_builder_fn=build_robot_from_args, world_loader_fn=world_loader_fn, **kwargs)
  File "/home/crslab/cehao/kitchen-worlds/pybullet_planning/problem_sets/problem_utils.py", line 87, in problem_template
    loaded_problem_dict = world_loader_fn(world, **world_builder_args)
  File "/home/crslab/cehao/kitchen-worlds/pybullet_planning/problem_sets/pr2_problems_pigi.py", line 44, in loader_fn
    goals, objects = sample_full_kitchen_goal_rearrange_to_storage(world, movables, counters)
  File "/home/crslab/cehao/kitchen-worlds/pybullet_planning/problem_sets/pr2_problems_pigi.py", line 165, in sample_full_kitchen_goal_rearrange_to_storage
    _, obj_bottom, objects = make_sure_obstacles(world, case, movables, counters, objects)
  File "/home/crslab/cehao/kitchen-worlds/pybullet_planning/world_builder/loaders_partnet_kitchen.py", line 1502, in make_sure_obstacles
    all_to_move += random.sample(objs, 2 if random.random() < 0.2 else 1)
  File "/home/crslab/.conda/envs/kitchen/lib/python3.8/random.py", line 363, in sample
    raise ValueError("Sample larger than population or is negative")
ValueError: Sample larger than population or is negative
CeHao-NUS commented 1 week ago

By the way, how do I disable the GUI display? It seems use_gui = True is hardcoded and cannot be easily adjusted in the argparser?

zt-yang commented 1 week ago

The problems with frozen screen is fixed. The problem happens when sim.simulate was set to True. Without physics on you can still generate the motion plans and objects will be assumed attached after gripper is closed.

As for the pddlstream error, could you make sure to pull the most recent changes from pddlstream caelan/diverse branch?

I don't see the error when running python your_project_folder/run_generation_pigi_custom.py

To disable GUI display, set sim.viewer = False in config yaml files. setting them in argparse should be similar.

Let me know if you have other questions.

CeHao1 commented 1 week ago

Dear Zhutian,

Great thanks for your help.

I basically know the reason for this bug.

1. The recursive github pulling failed.

git clone git@github.com:Learning-and-Intelligent-Systems/kitchen-worlds.git --recursive

I tried to use the recursive clone function to download all sub-directories, but it failed.

git clone git@github.com:Learning-and-Intelligent-Systems/kitchen-worlds.git --recursive
Cloning into 'kitchen-worlds'...
ssh: connect to host github.com port 22: Connection timed out
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

So I changed the script, then I can download the kitchen-world repo, but the submodules were timeout.

git clone https://github.com/Learning-and-Intelligent-Systems/kitchen-worlds.git --recursive
Cloning into 'kitchen-worlds'...

remote: Enumerating objects: 11162, done.
remote: Counting objects: 100% (1465/1465), done.
remote: Compressing objects: 100% (515/515), done.
remote: Total 11162 (delta 935), reused 1441 (delta 919), pack-reused 9697 (from 1)
Receiving objects: 100% (11162/11162), 211.13 MiB | 11.20 MiB/s, done.
Resolving deltas: 100% (3375/3375), done.
Submodule 'assets/models' (git@github.com:zt-yang/kitchen-models.git) registered for path 'assets/models'
Submodule 'lisdf' (https://github.com/Learning-and-Intelligent-Systems/lisdf) registered for path 'lisdf'
Submodule 'pddlstream' (git@github.com:zt-yang/pddlstream.git) registered for path 'pddlstream'
Submodule 'pybullet_planning' (git@github.com:zt-yang/pybullet_planning.git) registered for path 'pybullet_planning'
Cloning into '/home/users/cehao/github_space/temp/kitchen-worlds/assets/models'...
ssh: connect to host github.com port 22: Connection timed out
fatal: Could not read from remote repository.

Cloning into '/home/users/cehao/github_space/temp/kitchen-worlds/lisdf'...
remote: Enumerating objects: 1064, done.        
remote: Counting objects: 100% (279/279), done.        
remote: Compressing objects: 100% (145/145), done.        
remote: Total 1064 (delta 152), reused 208 (delta 123), pack-reused 785 (from 1)        
Receiving objects: 100% (1064/1064), 6.83 MiB | 11.29 MiB/s, done.
Resolving deltas: 100% (563/563), done.
Cloning into '/home/users/cehao/github_space/temp/kitchen-worlds/pddlstream'...
ssh: connect to host github.com port 22: Connection timed out
fatal: Could not read from remote repository.

2. Therefore, I manually download the submodules. Then put them into the correct directories.

However, some repos are not specific branches. link

So I need to manually switch to the exact commit. And this might make some mismatch with your local version.

3. I will double-check all repos again and tell you if it can work.

If you have time, would you help check how to successfully download the submodules together? To see whether this can work? git clone git@github.com:Learning-and-Intelligent-Systems/kitchen-worlds.git --recursive

Great thanks for your help. And I want to develop some transformer and diffusion-based TAMP algorithms!

Best wishes, Ce Hao