tjarkvandemerwe / tidyprompt

Create LLM prompting pipelines
https://tjarkvandemerwe.github.io/tidyprompt/
Other
3 stars 1 forks source link

send_prompt: wrap-by-wrap implementation + cleanup arguments #14

Closed tjarkvandemerwe closed 4 weeks ago

tjarkvandemerwe commented 1 month ago
send_prompt <- function(
    prompt,
    llm_provider = NULL,
    system_prompt = NULL,
    extraction_functions = list(),
    validation_functions = list(),
    max_retries = 10,
    verbose = getOption("tidyprompt.verbose", FALSE),
    extract_validate_mode = c("extraction_then_validation", "wrap_by_wrap")
)

Remove extraction_functions and validation_functions from send prompt. These are always present in the prompt object.

llm_provider: can this be an option aswell? And remove the option to add llm_provider to prompt, you only need this when sending a prompt.

extract_validate_mode: I think there is only 1 right way to do this, to make the right validation function (and return error message) go off. For instance, when you have wrapper A and B. And the extract from A and B go off first. Maybe the extract from B removes a part of A that needs to be validated. And thus this validation is unable to run. e.g.:

# Retrieves the first letters of forest animals:
'can you give me a list of animals in the forest?' |>
  answer_as_list() |>
  prompt_wrap(validate_forest_animals) |>
  answer_as_single_letter(n = 1, 'first')

Retrieving the first letters prevents the validation of forest animals.

lukakoning commented 1 month ago

Agreed that we can remove options to create one right way to perform actions. Llm_provider should indeed probably be a part of send_prompt, others should be part of the prompt itself.

I would retain the current mode but indeed set the 'wrap by wrap' mode as the new default.

lukakoning commented 1 month ago

WIP, done pending wrap-by-wrap implementation

lukakoning commented 4 weeks ago

Dit heb ik gedaan en inmiddels gemerged met main

Ik heb er nog voor gekozen om toch 'max_interactions' aan send_prompt toe te voegen; om goed layer-by-layer te doen vergt wat complexe code (gegeven dat je bijv. de wrap met tool-calls altijd beschikbaar wil houden, ook voor eventuele tussentijdse tool calls). In praktijk dacht ik dat uiteindelijk de user het vooral gaat om een prompt completen binnen x aantal interacties, en dat het dan niet per se uitmaakt hoe de verdeling van interacties is.