Specifying remote server doesn't work with query()

[bshor]

I have a remote server running Ollama. I can list the models just fine with:

list_models(server="http://server:11434")

But

query("why is the sky blue?",server=server_url)

fails with

Error in `httr2::req_perform()`:
! Failed to perform HTTP request.
Caused by error in `curl::curl_fetch_memory()`:
! Failed to connect to localhost port 11434 after 0 ms: Couldn't connect to server

Looks like it still defaults to localhost even when a different server is specified.

[JBGruber]

Thanks for notifying us! I rewrote the backend for the query requests and I think I fixed it. At least it definitly works now. Please install the development version of rollama and try again.

[bshor]

Thank you for following up! Unfortunately, it's not fixed -- I get the same error with query (but ping and list_model do work). Probably I think this is because the development version hasn't advanced? I uninstalled the CRAN version and installed the Github version with remotes, but it's still 0.1.0?

── R CMD build ───────────────────────────────────────────────────────────────────────────────────────────
✔  checking for file ‘/tmp/Rtmpw9NpRS/remotesd3d119d47016/JBGruber-rollama-7b2d88b/DESCRIPTION’ ...
─  preparing ‘rollama’:
✔  checking DESCRIPTION meta-information
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘rollama_0.1.0.9000.tar.gz’

[JBGruber]

The version number should be 0.1.0.9000. Installing with remotes should do the trick. After installation, you need to restart R. If it still does not work, could you produce a reproducible example? I.e., share the entire code you are running when you get the error. There is a nice tool for that called reprex.

[bshor]

Some progress, perhaps. I hadn't restarted R since I uninstalled rollama and reinstalled the 9000 version with remotes.

Here's my ridiculously simple reprex (with the exception of the server_url which I guess will be different for you).

library(rollama)

server_url="http://zen4:11434"
ping_ollama(server=server_url)
query(q="why is the sky blue?",server=server_url, model="llama3.2:3b-instruct-q4_K_M")
list_models(server=server_url)

And the results show a different error on query, of a model not found. But I definitely have this model.

> ping_ollama(server=server_url)
▶ Ollama (v0.3.12) is running at <http://zen4:11434>!
> query(q="why is the sky blue?",server=server_url, model="llama3.2:3b-instruct-q4_K_M")
Error in perform_reqs(reqs, verbose) : object 'model' not found

List_models works (but with a little delay).

list_models(server=server_url)
# A tibble: 16 × 11
   name                model modified_at   size digest parent_model format family families parameter_size
   <chr>               <chr> <chr>        <dbl> <chr>  <chr>        <chr>  <chr>  <list>   <chr>         
 1 llama3.2:3b-instru… llam… 2024-09-26… 2.02e9 a80c4… ""           gguf   llama  <chr>    3.2B          
 2 mistral:7b-instruc… mist… 2024-09-25… 4.37e9 1a856… ""           gguf   llama  <chr>    7B            
 3 nemotron-mini:4b-i… nemo… 2024-09-22… 2.70e9 ed76a… ""           gguf   nemot… <chr>    4.2B         

Happy to do more involved stuff to help like reprex, but this is a few lines of code.

FYI, on my client (laptop) I'm running R 4.4.1 on Fedora 40 (technically a Fedora 40 distrobox running in uBlue Aurora 40, which is an immutable Fedora distribution like silverblue/kinoite), everything updated. Remote server zen4 (my desktop) is running R 4.4.1 on Fedora 40 as well. Rollama on the desktop works perfectly!

[JBGruber]

I made a small dumb mistake commiting the last update. But it works now! Thanks for the example code:

remotes::install_github("JBGruber/rollama")

Using GitHub PAT from the git credential store.

Skipping install of 'rollama' from a github remote, the SHA1 (b2587f55) has not changed since last install.
  Use `force = TRUE` to force installation

library(rollama)

ping_ollama()

✖ Could not connect to Ollama at <http://localhost:11434>

server_url="http://192.168.2.29:11434"
ping_ollama(server=server_url)

▶ Ollama (v0.3.12) is running at <http://192.168.2.29:11434>!

query(q="why is the sky blue?",server=server_url, model="llama3.2:3b-instruct-q4_K_M")

── Answer from llama3.2:3b-instruct-q4_K_M ─────────────────────────────────────

The sky appears blue to our eyes because of a phenomenon called Rayleigh
scattering, named after the British physicist Lord Rayleigh, who first
explained it in the late 19th century.

Here's what happens:

1. When sunlight enters Earth's atmosphere, it encounters tiny molecules of
gases such as nitrogen (N2) and oxygen (O2).

2. These molecules scatter the light in all directions, but they scatter
shorter (blue) wavelengths more than longer (red) wavelengths.

3. This is because the smaller molecules are more effective at scattering the
shorter wavelengths.

4. As a result, the blue light is scattered in all directions and reaches our
eyes from all parts of the sky.

5. Our eyes perceive this scattered blue light as the color of the sky.

The reason why the sky isn't just blue, but also has hues of gray, white, and
sometimes even red or orange, is because of other factors:

* Dust, water vapor, and pollutants in the air can scatter light in different
ways, adding to the color of the sky.

* The angle at which the sunlight enters the atmosphere and reaches our eyes
affects the apparent color of the sky.

* The time of day, atmospheric conditions, and weather patterns can also
influence the appearance of the sky.

So, to summarize: the sky appears blue because of Rayleigh scattering, which
scatters shorter (blue) wavelengths of light more than longer (red)
wavelengths.

list_models(server=server_url)

# A tibble: 7 × 11
  name      model modified_at    size digest parent_model format family families
  <chr>     <chr> <chr>         <dbl> <chr>  <chr>        <chr>  <chr>  <list>  
1 llama3.1… llam… 2024-10-09… 4.66e 9 42182… ""           gguf   llama  <chr>   
2 llama3.2… llam… 2024-10-09… 2.02e 9 a80c4… ""           gguf   llama  <chr>   
3 nomic-em… nomi… 2024-10-01… 2.74e 8 0a109… ""           gguf   nomic… <chr>   
4 llava:la… llav… 2024-10-01… 4.73e 9 8dd30… ""           gguf   llama  <chr>   
5 llava:la… llav… 2024-10-01… 4.73e 9 8dd30… ""           gguf   llama  <chr>   
6 llama3:l… llam… 2024-10-01… 4.66e 9 365c0… ""           gguf   llama  <chr>   
7 mixtral:… mixt… 2024-10-01… 2.64e10 d39eb… ""           gguf   llama  <chr>   
# ℹ 2 more variables: parameter_size <chr>, quantization_level <chr>

[bshor]

Excellent! It works!