BatsResearch / planetarium

Dataset and benchmark for assessing LLMs in translating natural language descriptions of planning problems into PDDL
BSD 3-Clause "New" or "Revised" License
42 stars 4 forks source link

Question on Scene Graph #8

Open harshakokel opened 3 months ago

harshakokel commented 3 months ago

Hello,

I am trying to understand how you evaluate the problem equivalence. It seems that the isomorphism in scene graphs of two states is like checking if predicate description of two states can be unified (if the objects were considered as variables). For example state 1: (clear a) (clear b) (on-table a) (on-table b) (arm-empty)) and state 2: (clear d) (clear e) (on-table e) (on-table d) (arm-empty)) can be unified as follows a -> d, b -> e. So they are equivalent. Is my understanding correct?

Also, from the section 3.3 in the paper, it seems that the graph contains edges between proposition type and objects, but in Fig 4 has an edge from clear to arm-empty. I am confused as to what that edge indicates.

Regards, Harsha Kokel

maxzuo commented 3 months ago

Hi,

Thanks for your interest in our work! I wasn't familiar with unification before, but yes, it seems at least from glancing that isomorphism of two scene graphs is like checking if the two states can be unified.

The one thing to add is we have to consider also adding "trivial" predicates before this step for goal conditions, so the goal (on-table f) (on-table g) is also equivalent, because the only state that can be reached from a normal, valid setup with these two blocks is (on-table f) (on-table g) (clear f) (clear g) (arm-empty)

Your understanding of the scene graphs is correct! What is displayed is a simplified graph representation (there's a little * and a blurb about it on the bottom right). A big part of making our code work well over the graphs is building a function that makes a reduced/compressed graph representation that is more computationally efficient to work with. In our code, we convert PDDL to a generic graph representation, and then into a reduced representation. The function is planetarium.oracle.reduce if you're interested.

Hope this helps!