azfeng8 / GLIB-Baking-Fails-and-LLMs

0 stars 0 forks source link

LLM operator library init #4

Closed azfeng8 closed 5 months ago

azfeng8 commented 5 months ago

PR request @NishanthJKumar to review

azfeng8 commented 5 months ago

Also seems to be an error on line 35 of openai_interface.py: the tuple in the type annotation should be Tuple. I notice this issue in a bunch of type annotations (using tuple instead of Tuple or list instead of List, and am surprised this runs without any errors). Those are pretty minor though.

The type hints are for reading only, I am not enforcing them, so maybe that's why they don't cause errors. I'll change the "tuple" to Tuple and so on going forward.

Overall, I think things look good, though do check the comments I left in the zpk_operator_learning file. I think as we discussed earlier, the main things to look into with this batch of experiments is how often we're getting useful transitions from in_plan operators, and also how many LLM-derived operators we have at various points (to try to understand if they're actually being used for planning and exploration much, or if they're just leading us astray in various ways)

Thanks for the review. I'll modify the data viz and logging and run the experiments. Probably will need more scripting tools.