Grammar-Based Prompt Format Optimization

[diegovalenzuelaiturra]

Hi,

As I’m learning about DSPy, I came across an interesting approach that uses grammar-based prompt format optimization. I’m curious if this is something already implemented in DSPy, or if the current method is considered more effective.

The method I found involves defining a prompt format optimization problem as a search over a space of semantically equivalent prompt formats defined by a grammar. They propose using Bayesian optimization to explore this space efficiently under a user-specified computational budget.

Here’s the paper for reference:

Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting

• Paper: https://arxiv.org/abs/2310.11324

• GitHub: https://github.com/msclar/formatspread

Thanks for your insights!

[arnavsinghvi11]

This might be better placed in Discussions, and feel free to explore the MIPROv2 optimizer which uses Bayesian optimization to produce optimized prompts - might be a useful application here or room for improvement to include in MIPRO!