tidyverse / elmer

Call LLM APIs from R
http://elmer.tidyverse.org/
Other
208 stars 30 forks source link

feat(contents_text): Add `contents_*()` helper functions #170

Open gadenbuie opened 1 week ago

gadenbuie commented 1 week ago

Fixes #167

Adds contents_text(), contents_markdown() and contents_html() generics that can be applied to Turn and Content objects.

turns <- list(
  user_turn("What's this image?", content_image_url("https://placehold.co/200x200")),
  Turn("assistant", "It's a placeholder image.")
)

lapply(turns, contents_text)
#> [[1]]
#> [1] "What's this image?"
#> 
#> [[2]]
#> [1] "It's a placeholder image."
lapply(turns, contents_markdown)
#> [[1]]
#> [1] "What's this image?\n\n![](https://placehold.co/200x200)"
#> 
#> [[2]]
#> [1] "It's a placeholder image."
contents_html(turns[[1]])
#> [1] "<p>What's this image?</p>\n\n<img src=\"https://placehold.co/200x200\">"
gadenbuie commented 1 week ago

@hadley this is ready for another look. Sorry for the commit noise, I only just now realized I wasn't able to run checks locally because I had out-of-date credentials. Of course when I finally got to ✅ in the last commit there was an unrelated CI failure and I can't request a rerun.

cpsievert commented 2 days ago

I love this idea in general, but also had a couple thoughts on this PR:

  1. It'd be nice if these content extractors had the ability to get "additional" content (i.e., the stuff you get with $chat(echo="all")), mainly for debugging (but also educational) purposes.
  2. It feels like the main way people would want to use this is to generate an md/html file of the whole conversation? For that reason, instead of generics, I think for chatlas I might do something like chat.export(filename="output.html", content="all"), which might internally call a new turn.get_content("all").