Text embedding slow compared to Python client

[JBGruber]

Example:

library(rollama)
library(tidyverse)
file <- "https://www.gutenberg.org/cache/epub/16/pg16.txt"

pp_df <- tibble(line = readLines(file)) |> 
  mutate(par = str_detect(line, "^\\s*$"),
         par_id = cumsum(par)) |> 
  filter(!par) |> 
  group_by(par_id) |> 
  summarise(paragraph = paste(line, collapse = " "))

pp_embeddings <- pp_df |> 
  mutate(embeddings = embed_text(text = paragraph, model = "nomic-embed-text")) |> 
  unnest_wider(embeddings)

The embedding step ran for 10m 40.2s (✔ embedded 1736 texts [10m 40.2s]) while the same task took 2m 45s from the Ollama python library. Not sure what's going on.

[JBGruber]

Now we're quite a bit faster :grin:: ✔ embedded 1736 texts [1m 26.9s]!

Technical background: I'm sending all texts to embed text at once via asynchronous requests. This essentially uses the queue of Ollama itself instead of queuing on the client side.