compiled prompts

[DmitriyG228]

Hey, while inspecting the compiled save json I was expecting to see the compiled prompts. I can see only "demos" and other fields just empty.

I am confused since the pipeline behaves as expected and seems compiled. Does this mean the pipe is just not compiled, or the actual prompts are stored in a more sophisticated way?

Thank you

[okhat]

The prompts are constructed from these pieces.

You can use turbo.inspect_history(n=3) after using your program to see the last n=3 prompts to your lm, eg, turbo

[andreapiso]

@okhat is there any way to extract the compiled prompts before the execution? e.g. still with their template keywords {question}, {search_query} etc... it would be useful to be able to remove DSPy as a dependency for production systems, where I would train a prompt with DSPy, extract it, and use it in my own inference server (of course, i would have to take care of injecting the template value at runtime).

[okhat]

Yeah you can do program.save(path)

[okhat]

Or, less clean but gives your the string prompt: my_lm.history[-1]['prompt'] and extract it from there....

[andreapiso]

Thanks a lot @okhat! I guess the program.save does not contain the prompt itself, but all the components that can be used to reconstruct it, while lm.history contains the prompt but with the user input already 'injected` so you'd have to scrape it and re-templatise it if you wanted to create string templates.

Similarly, how would one go about getting the prompts for the other "rejected" candidate programs? Would the best way just go trough the entire lm.history and build some logic to understand the different permutations of demonstrations, instructions, descriptions etc.?

[okhat]

To get rejected candidate programs, simply access compiled_program.candidate_programs. They're sorted from best to worst!

[andreapiso]

Interesting! Are the candidate programs right now iterating only through different combinations of demonstrations, or are there modifiers on the prompt itself? (instructions, descriptions, etc...)

[okhat]

The teleprompters that are on the main branch now create new demonstrations and sample from them them.

We have other work (ongoing) on optimizing the instructions and descriptions, but not ready yet.

Importantly, the output values for the demonstrations in BootstrapFewShot and others are created from scratch, they're not just selected from your data.