xavierpuigf / virtualhome

API to run VirtualHome, a Multi-Agent Household Simulator
http://virtual-home.org
MIT License
454 stars 65 forks source link

How to judge whether the action is executed correctly? #88

Open Xianqi-Zhang opened 1 year ago

Xianqi-Zhang commented 1 year ago

Hi, I have 2 problems.

  1. How to judge whether the action is executed correctly?

First, add a cat in one of the sofas and a character in the kitchen.

success, graph = comm.environment_graph();
sofa = find_nodes(graph, class_name='sofa')[-2]
print(sofa)
add_node(graph, {'class_name': 'cat', 
                   'category': 'Animals', 
                   'id': 1000, 
                   'properties': [], 
                   'states': []})
add_edge(graph, 1000, 'ON', sofa['id'])

comm.reset(1)
tv_node['states'] = ['OFF']
comm.expand_scene(prev_graph)
comm.add_character('chars/Male1', initial_room='kitchen')
s, g = comm.environment_graph()
cat_id = [node['id'] for node in g['nodes'] if node['class_name'] == 'cat'][0]

After that, directly execute action : sit sofa and walk tv

script = [ '<char0> [Sit] <sofa> ({})'.format(sofa['id'])]
success, message = comm.render_script(script=script,
                                      processing_time_limit=60,
                                      find_solution=False,
                                      skip_execution=True, # * !!!
                                      skip_animation=True, # * !!!
                                      recording=True,
                                      save_pose_data=False)
Output: 
success
True
message
{'0': {'message': 'Success'}}

script = ['<char0> [[Walk]] <tv> ({})'.format(tv_node['id'])]
success, message = comm.render_script(script=script,
                                      processing_time_limit=60,
                                      find_solution=False,
                                      skip_execution=True, # * !!!
                                      skip_animation=True, # * !!!
                                      recording=True,
                                      save_pose_data=False)
Output: 
success
True
message
{'0': {'message': 'Success'}}

Without Walk sofa and StandUp, but Walk tv is not right because it doesn't meet the preconditions of the walk (character is not sitting). http://virtual-home.org/documentation/master/kb/actions.html

So, how to judge whether an action is executed correctly?

  1. I find that if the call is wrong, such as performing an invalid action (Grab sofa), the executable program will crash. How to explore the environment automatically?

Virtual home v2.2.4

  1. It seems that links of (VirtualHome Activity) and (ActivityPrograms) should be exchanged. (Readme file: Original Collected and Scripted Programs) VirtualHome Activity shoule be synthetic. ActivityPrograms should be real.

Thank you for your any replay.