bosquet.llm.openai/->error fails when both if branches return nil

[ash-mcc]

Hi Žygimantas, a minor thing... bosquet.llm.openai/->error fails (with the message Additional data must be non-nil.) when both if its if branch return nil.

I came across this scenario when I was trying to use a local Llama2 service (with a OpenAI API). but I had incorrect values in system.edn ...which resulted in wkok.openai-clojure.core throwing something that caused the scenario in bosquet.llm.openai/->error.

Cheers, Ash

[zmedelis]

@ash-mcc If you can, please check https://github.com/zmedelis/bosquet/commit/4d106d6c09f074e5b7f23f8214f4295e0cc50172 and try running it on your failing setup.

I can not properly reproduce this, but the commit tries to address it not fail, and log the underlying cause. If you manage to get it into that failing path, please send me what was logged.

[ash-mcc]

Thanks @zmedelis Your https://github.com/zmedelis/bosquet/commit/4d106d6c09f074e5b7f23f8214f4295e0cc50172 commit has fixed this issue.

I replicated the failure scenario by setting a bad :impl value in system.edn, e.g.

 ;; Config for an OpenAI API fronted LLM provided by a local Llama2 based service
 [:llm/openai :provider/llama2] {:api-key            #ref [:config :llama2-openai-api-key]
                                 :api-endpoint       #ref [:config :llama2-openai-api-endpoint]
                                 :impl               :should-be-openai}

And in this scenario, your new code now outputs the much more helpful:

2023-11-21T16:55:22.601Z zinc.local INFO [bosquet.llm.openai:112] -     * Options: '{:impl :should-be-openai, :api-endpoint "http://localhost:8000/v1"}'
; 2023-11-21T16:55:22.603Z zinc.local ERROR [bosquet.llm.openai:92] - Error when making OAI call. Error data: #error {
;  :cause "No matching clause: :should-be-openai"
;  :via
;  [{:type java.lang.IllegalArgumentException
;    :message "No matching clause: :should-be-openai"
;    :at [wkok.openai_clojure.core$response_for invokeStatic "core.clj" 26]}]
;  :trace
;  [[wkok.openai_clojure.core$response_for invokeStatic "core.clj" 26]
;   [wkok.openai_clojure.core$response_for invoke "core.clj" 16]
;   [wkok.openai_clojure.api$create_chat_completion invokeStatic "api.clj" 84]
;   [wkok.openai_clojure.api$create_chat_completion invoke "api.clj" 63]
;   [bosquet.llm.openai$create_chat_completion invokeStatic "openai.clj" 63]
;   [bosquet.llm.openai$create_chat_completion invoke "openai.clj" 57]
;   [bosquet.llm.openai$complete invokeStatic "openai.clj" 115]
;   [bosquet.llm.openai$complete invoke "openai.clj" 95]
;   [bosquet.llm.openai.OpenAI generate "openai.clj" 151]
;   [jdk.internal.reflect.NativeMethodAccessorImpl invoke0 "NativeMethodAccess
; orImpl.java" -2]
;   [jdk.internal.reflect.NativeMethodAccessorImpl invoke "NativeMethodAccessorImpl.java" 77]
;   [jdk.internal.reflect.DelegatingMethodAccessorImpl invoke "DelegatingMethodAccessorImpl.java" 43]
;   [java.lang.reflect.Method invoke "Method.java" 568]
;   [clojure.lang.Reflector invokeMatchingMethod "Reflector.java" 167]
;   [clojure.lang.Reflector invokeInstanceMethod "Reflector.java" 102]
;   [adapt.vfgreact.x2_bosquet_simple_gen$eval36772 invokeStatic "NO_SOURCE_FILE" 31]
;   [adapt.vfgreact.x2_bosquet_simple_gen$eval36772 invoke "NO_SOURCE_FILE" 31]
;   [clojure.lang.Compiler eval "Compiler.java" 7194]
;   [clojure.lang.Compiler eval "Compiler.java" 7149]
;   [clojure.core$eval invokeStatic "core.clj" 3215]
;   [clojure.core$eval invoke "core.clj" 3211]
;   [nrepl.middleware.interruptible_eval$evaluate$fn__1265$fn__1266 invoke "interruptible_eval.clj" 87]
;   [clojure.lang.AFn applyToHelper "AFn.java" 152]
;   [clojure.lang.AFn applyTo "AFn.java" 144]
;   [clojure.core$apply invokeStatic "core.clj" 667]
;   [clojure.cor
; e$with_bindings_STAR_ invokeStatic "core.clj" 1990]
;   [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1990]
;   [clojure.lang.RestFn invoke "RestFn.java" 425]
;   [nrepl.middleware.interruptible_eval$evaluate$fn__1265 invoke "interruptible_eval.clj" 87]
;   [clojure.main$repl$read_eval_print__9206$fn__9209 invoke "main.clj" 437]
;   [clojure.main$repl$read_eval_print__9206 invoke "main.clj" 437]
;   [clojure.main$repl$fn__9215 invoke "main.clj" 458]
;   [clojure.main$repl invokeStatic "main.clj" 458]
;   [clojure.main$repl doInvoke "main.clj" 368]
;   [clojure.lang.RestFn invoke "RestFn.java" 1523]
;   [nrepl.middleware.interruptible_eval$evaluate invokeStatic "interruptible_eval.clj" 84]
;   [nrepl.middleware.interruptible_eval$evaluate invoke "interruptible_eval.clj" 56]
;   [nrepl.middleware.interruptible_eval$interruptible_eval$fn__1298$fn__1302 invoke "interruptible_eval.clj" 152]
;   [clojure.lang.AFn run "AFn.java" 22]
;   [nrepl.middleware.session$session_exec$main_loop__1368$fn__1372 invoke "session.clj" 218]
;   [nrepl.m
; iddleware.session$session_exec$main_loop__1368 invoke "session.clj" 217]
;   [clojure.lang.AFn run "AFn.java" 22]
;   [java.lang.Thread run "Thread.java" 833]]}
; 
; Execution error (ExceptionInfo) at bosquet.llm.openai/->error (openai.clj:81).
; Completion error in OAI call

So, thank you. Cheers, Ash

[zmedelis]

Now part of the latest release.

There is another interesting aspect. You are adding your system component. Now system.edn is hidden from modification. You probably need to do things like with-redef, fork the lib, or something not straightforward to add new components.

See #44 on my intentions for this matter.