Prompt changes

[TathagataChakraborti]

• [x] For the slot filler line, the second line should go after the third it sounds more natural coz it says what ask does first before saying its less preferred.

  • The system has Action ask. Action ask is less preferred than acquiring the value of a variable through other actions. Action ask lets the system ask the user about the value of any variable unless that variable is explicitly marked as cannot be acquired from the user.
  • versus
  • The system has Action ask. Action ask lets the system ask the user about the value of any variable unless that variable is explicitly marked as cannot be acquired from the user. Action ask is less preferred than acquiring the value of a variable through other actions.

• [x] The slot fillable line should come after the introduction of the ask operation. The current description says which things cannot be asked before introducing the ask operation.

• [x] We should probably move the slot filler description to after the actions, next to or even after the map description.

  • This puts the action description immediately after the introduction of the system has action A and B.
  • This also makes the "through other actions" part make more sense coz some other actions has already been introduced.
  • Optionally, we can also explicitly mention other actions as mapping and system actions.

• [x] In the action description, add a line like action A has parameters x, y, z before ... To execute action A, variable x, y, z should be known.

  • In the llm plan sometimes it is missing the parameters i think this might fix it

• [x] Change the mapping line from Values for Variable v1 can be used for Variable v0 to include the map operation. Something like Action map can determine the value of Variable v1 from Variable v0.

  • For the problems with items in memory the llms is sometimes skilling the map operation, maybe it assumes from the current sentence that the new values are automatically determined, this might fix it.

• [x] Uncapitalize action, variable, etc. Currently some are capitalized, some are not.

• [x] ~Remove double underscores?~

Current

The system has Action a__0 and Action a__1.
Variables v__0, v__1, and v__2 cannot be acquired by asking the user.
The system has Action ask. Action ask is less preferred than acquiring the value of a variable through other actions. Action ask lets the system ask the user about the value of any variable unless that variable is explicitly marked as cannot be acquired from the user.
To execute Action a__1, Variable v__2 should be known.
After executing Action a__1, Variable v__1 is known.
To execute Action a__0, Variable v__0 should be known.
After executing Action a__0, Variable v__2 is known.
Values are available already for Variable v__3.
Action map is used when a value for one variable can be used for another variable.
Values for Variable v__1 can be used for Variable v__0. Values for Variable v__0 can be used for Variable v__1. Values for Variable v__0 can be used for Variable v__3. Values for Variable v__3 can be used for Variable v__0.
The goal of the system is to execute Action a__1.

Desired

The system has action a_0 and action a_1.
Action a_1 has parameters v_2 and outputs v_1.
To execute action a_1, variable v_2 should be known.
After executing action a_1, variable v_1 is known.
Action a_0 has parameters v_0 and outputs v_2.
To execute action a_0, variable v_0 should be known.
After executing action a_0, variable v_2 is known.
Values are available already for variable v_3.
Action map is used when a value for one variable can be used for another variable.
Using action map, values for variable v__1 can be determined from variable v__0. Using action map, values for variable v__0 can be determined from variable v__1. Using action map, values for variable v__0 can be determined from variable v__3. Using action map, values for variable v__3 can be determined from variable v__0. 
The system has action ask. Action ask lets the system ask the user about the value of any variable unless that variable is explicitly marked as cannot be acquired from the user. Action ask is less preferred than acquiring the value of a variable through other actions like system actions and mapping. 
Variables v_0, v_1, and v_2 cannot be acquired by asking the user.
The goal of the system is to execute action a_1.

Circular constraints

I noticed that many samples have this pattern: the parameters are cycled through in order and back again.

        a__0 = Operator("a__0")
        a__0.add_input(SignatureItem(parameters="v__0"))
        a__0.add_output(SignatureItem(parameters="v__2"))

        a__1 = Operator("a__1")
        a__1.add_input(SignatureItem(parameters="v__2"))
        a__1.add_output(SignatureItem(parameters="v__1"))

        a__0 = Operator("a__0")
        a__0.add_input(SignatureItem(parameters="v__0"))
        a__0.add_output(SignatureItem(parameters="v__2"))

        a__1 = Operator("a__1")
        a__1.add_input(SignatureItem(parameters="v__2"))
        a__1.add_output(SignatureItem(parameters="v__3"))

        a__2 = Operator("a__2")
        a__2.add_input(SignatureItem(parameters="v__3"))
        a__2.add_output(SignatureItem(parameters="v__1"))

This might impact the performance by being biased to a particular class of problems where inputs and outputs are circular. Can we randomize this? Or are other places randomized so it doesn't matter?

[jkeskingvillage]

@TathagataChakraborti Circular constraints happen due to the nature of algorithm to create coupling. We can re-visit this if large number of actions is needed. Meanwhile, this is, also, not rare in the real world APIs (i.e. id, name, token, time, ...).

[jkeskingvillage]

@TathagataChakraborti Let's leave double underscores in names as they are. This can be useful for parsing responses from LLMs. Everything else will be done.

[jkeskingvillage]

Done.

[jkeskingvillage]

Some more changes are needed.

[jkeskingvillage]

Example planning problem descriptions. planning_examples.txt