ggml-org / llama.vim

Vim plugin for LLM-assisted code/text completion
85 stars 4 forks source link

add_quotations #9

Closed PhilippKaesgen closed 6 hours ago

PhilippKaesgen commented 6 hours ago

The quotation marks around l:request are important when sending the curl_command through ssh, for instance, as in:

let l:curl_command = [
        \ "ssh", "-q",
        \ "-J", "jump.server",
        \ "llama.cpp.server",
        \ "curl",
        \ "--silent",
        \ "--no-buffer",
        \ "--request", "POST",
        \ "--url", g:llama_config.endpoint,
        \ "--header", "Content-Type: application/json",
        \ "--data", "'". l:request ."'"
        \ ]

Without the quotation marks, the ssh command might fail, hence this PR.

Why would you need that?

On a server cluster, you might want to run the llama.cpp server on one server and work on a file in vim on another server. Then you need to route your curl command to the llama.cpp server (possibly via a jump server) to use the curl command as is.

This should not affect the generality of the curl_command. Putting this here because it took me a while to figure out.

ggerganov commented 6 hours ago

Instead of modifying the curl command, I forward the remote port to a local port:

ssh -fNL 8012:localhost:8012 <remote_host>
m18coppola commented 5 hours ago

This broke functionality for me:

got exception: {"code":500,"message":"[json.exception.parse_error.101] parse error at line 1, column 1: syntax
error while parsing value - invalid literal; last read: '''","type":"server_error"}
m18coppola commented 5 hours ago

From vim's job_start help page:

The command is executed directly, not through a shell, the
                'shell' option is not used.  To use the shell:
        let job = job_start(["/bin/sh", "-c", "echo hello"])
                Or:
        let job = job_start('/bin/sh -c "echo hello"')
                Note that this will start two processes, the shell and the
                command it executes.  If you don't want this use the "exec"
                shell command.

From neovim's jobstart help page:

        Spawns {cmd} as a job.
        If {cmd} is a List it runs directly (no 'shell').

There's no need to wrap it in single quotes, as there is no shell it misinterpret escape characters. The only reason why you needed to add single quotes was because you introduced a shell by using the ssh command. I recommend you use port-forwarding like @ggerganov recommended.

ggerganov commented 5 hours ago

Ah sorry about that. I've reverted the change.