search

wkok / openai-clojure

Clojure functions to drive the OpenAI API
https://cljdoc.org/d/net.clojars.wkok/openai-clojure
MIT License
208 stars 28 forks source link

ClassCastException when passing in request option async? #52

Closed nsadeh closed 9 months ago

nsadeh commented 10 months ago

Description

When passing request option async? as described in the documentation, I get a ClassCastException: class jdk.internal.net.http.common.MinimalFuture cannot be cast to class clojure.lang.Associative. This is traced back to the function response-for in core.

My guess is that there's some code that does something with the response and assumes that it is always a map/Associative when it isn't streaming. I use and like this library enough that I would be happy to fix if you have some initial direction for where the problem is.

Reproduction

I ran the following query:

> (api/create-chat-completion {:model "gpt-3.5-turbo"
                                                  :messages [{:content "PING" :role "user"}]
                                                  :temperature 0
                                                  }
                                                 {:api-key openai-token
                                                  :request {:async? true}})

And got the following stack trace:

Unhandled clojure.lang.ExceptionInfo
   Interceptor Exception: class
   jdk.internal.net.http.common.MinimalFuture cannot be cast to class
   clojure.lang.Associative
   (jdk.internal.net.http.common.MinimalFuture is in module
   java.net.http of loader 'platform'; clojure.lang.Associative is in
   unnamed module of loader 'app')
   {:execution-id #uuid "2cdd723b-2410-4795-ac27-e28dff70677e",
    :stage :leave,
    :interceptor :martian.hato/keywordize-headers,
    :type java.lang.ClassCastException,
    :exception #error {
    :cause "class jdk.internal.net.http.common.MinimalFuture cannot be cast to class clojure.lang.Associative (jdk.internal.net.http.common.MinimalFuture is in module java.net.http of loader 'platform'; clojure.lang.Associative is in unnamed module of loader 'app')"
    :via
    [{:type java.lang.ClassCastException
      :message "class jdk.internal.net.http.common.MinimalFuture cannot be cast to class clojure.lang.Associative (jdk.internal.net.http.common.MinimalFuture is in module java.net.http of loader 'platform'; clojure.lang.Associative is in unnamed module of loader 'app')"
      :at [clojure.lang.RT assoc "RT.java" 827]}]
    :trace
    [[clojure.lang.RT assoc "RT.java" 827]
     [clojure.core$assoc__5481 invokeStatic "core.clj" 193]
     [clojure.core$update_in$up__6922 invoke "core.clj" 6220]
     [clojure.core$update_in$up__6922 invoke "core.clj" 6219]
     [clojure.core$update_in invokeStatic "core.clj" 6221]
     [clojure.core$update_in doInvoke "core.clj" 6207]
     [clojure.lang.RestFn invoke "RestFn.java" 445]
     [martian.hato$fn__18256 invokeStatic "hato.clj" 42]
     [martian.hato$fn__18256 invoke "hato.clj" 41]
     [tripod.context$try_f invokeStatic "context.cljc" 32]
     [tripod.context$try_f invoke "context.cljc" 22]
     [tripod.context$leave_all_with_binding invokeStatic "context.cljc" 136]
     [tripod.context$leave_all_with_binding invoke "context.cljc" 120]
     [tripod.context$leave_all$fn__13547 invoke "context.cljc" 149]
     [clojure.lang.AFn applyToHelper "AFn.java" 152]
     [clojure.lang.AFn applyTo "AFn.java" 144]
     [clojure.core$apply invokeStatic "core.clj" 667]
     [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1990]
     [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1990]
     [clojure.lang.RestFn invoke "RestFn.java" 425]
     [tripod.context$leave_all invokeStatic "context.cljc" 148]
     [tripod.context$leave_all invoke "context.cljc" 142]
     [tripod.context$execute invokeStatic "context.cljc" 199]
     [tripod.context$execute invoke "context.cljc" 198]
     [martian.core$response_for invokeStatic "core.cljc" 113]
     [martian.core$response_for invoke "core.cljc" 107]
     [wkok.openai_clojure.core$response_for invokeStatic "core.clj" 36]
     [wkok.openai_clojure.core$response_for invoke "core.clj" 16]
     [wkok.openai_clojure.api$create_chat_completion invokeStatic "api.clj" 84]
     [wkok.openai_clojure.api$create_chat_completion invoke "api.clj" 63]
     [server.conversation$eval37315 invokeStatic "form-init10524477006413499415.clj" 286]
     [server.conversation$eval37315 invoke "form-init10524477006413499415.clj" 286]
     [clojure.lang.Compiler eval "Compiler.java" 7194]
     [clojure.lang.Compiler eval "Compiler.java" 7149]
     [clojure.core$eval invokeStatic "core.clj" 3215]
     [clojure.core$eval invoke "core.clj" 3211]
     [nrepl.middleware.interruptible_eval$evaluate$fn__26957$fn__26958 invoke "interruptible_eval.clj" 87]
     [clojure.lang.AFn applyToHelper "AFn.java" 152]
     [clojure.lang.AFn applyTo "AFn.java" 144]
     [clojure.core$apply invokeStatic "core.clj" 667]
     [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1990]
     [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1990]
     [clojure.lang.RestFn invoke "RestFn.java" 425]
     [nrepl.middleware.interruptible_eval$evaluate$fn__26957 invoke "interruptible_eval.clj" 87]
     [clojure.main$repl$read_eval_print__9206$fn__9209 invoke "main.clj" 437]
     [clojure.main$repl$read_eval_print__9206 invoke "main.clj" 437]
     [clojure.main$repl$fn__9215 invoke "main.clj" 458]
     [clojure.main$repl invokeStatic "main.clj" 458]
     [clojure.main$repl doInvoke "main.clj" 368]
     [clojure.lang.RestFn invoke "RestFn.java" 1523]
     [nrepl.middleware.interruptible_eval$evaluate invokeStatic "interruptible_eval.clj" 84]
     [nrepl.middleware.interruptible_eval$evaluate invoke "interruptible_eval.clj" 56]
     [nrepl.middleware.interruptible_eval$interruptible_eval$fn__26990$fn__26994 invoke "interruptible_eval.clj" 152]
     [clojure.lang.AFn run "AFn.java" 22]
     [nrepl.middleware.session$session_exec$main_loop__27060$fn__27064 invoke "session.clj" 218]
     [nrepl.middleware.session$session_exec$main_loop__27060 invoke "session.clj" 217]
     [clojure.lang.AFn run "AFn.java" 22]
     [java.lang.Thread run "Thread.java" 1583]]}}
              context.cljc:   12  tripod.context$exception__GT_ex_info/invokeStatic
              context.cljc:   11  tripod.context$exception__GT_ex_info/invoke
              context.cljc:   35  tripod.context$try_f/invokeStatic
              context.cljc:   22  tripod.context$try_f/invoke
              context.cljc:  136  tripod.context$leave_all_with_binding/invokeStatic
              context.cljc:  120  tripod.context$leave_all_with_binding/invoke
              context.cljc:  149  tripod.context$leave_all$fn__13547/invoke
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  667  clojure.core/apply
                  core.clj: 1990  clojure.core/with-bindings*
                  core.clj: 1990  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
              context.cljc:  148  tripod.context$leave_all/invokeStatic
              context.cljc:  142  tripod.context$leave_all/invoke
              context.cljc:  199  tripod.context$execute/invokeStatic
              context.cljc:  198  tripod.context$execute/invoke
                 core.cljc:  113  martian.core$response_for/invokeStatic
                 core.cljc:  107  martian.core$response_for/invoke
                  core.clj:   36  wkok.openai-clojure.core/response-for
                  core.clj:   16  wkok.openai-clojure.core/response-for
                   api.clj:   84  wkok.openai-clojure.api/create-chat-completion
                   api.clj:   63  wkok.openai-clojure.api/create-chat-completion
                      REPL:  286  server.conversation/eval37315
                      REPL:  286  server.conversation/eval37315
             Compiler.java: 7194  clojure.lang.Compiler/eval
             Compiler.java: 7149  clojure.lang.Compiler/eval
                  core.clj: 3215  clojure.core/eval
                  core.clj: 3211  clojure.core/eval
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  667  clojure.core/apply
                  core.clj: 1990  clojure.core/with-bindings*
                  core.clj: 1990  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  437  clojure.main/repl/read-eval-print/fn
                  main.clj:  437  clojure.main/repl/read-eval-print
                  main.clj:  458  clojure.main/repl/fn
                  main.clj:  458  clojure.main/repl
                  main.clj:  368  clojure.main/repl
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  218  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  217  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java: 1583  java.lang.Thread/run

1. Caused by java.lang.ClassCastException
   class jdk.internal.net.http.common.MinimalFuture cannot be cast to
   class clojure.lang.Associative
   (jdk.internal.net.http.common.MinimalFuture is in module
   java.net.http of loader 'platform'; clojure.lang.Associative is in
   unnamed module of loader 'app')

                   RT.java:  827  clojure.lang.RT/assoc
                  core.clj:  193  clojure.core/assoc
                  core.clj: 6220  clojure.core/update-in/up
                  core.clj: 6219  clojure.core/update-in/up
                  core.clj: 6221  clojure.core/update-in
                  core.clj: 6207  clojure.core/update-in
               RestFn.java:  445  clojure.lang.RestFn/invoke
                  hato.clj:   42  martian.hato/fn
                  hato.clj:   41  martian.hato/fn
              context.cljc:   32  tripod.context$try_f/invokeStatic
              context.cljc:   22  tripod.context$try_f/invoke
              context.cljc:  136  tripod.context$leave_all_with_binding/invokeStatic
              context.cljc:  120  tripod.context$leave_all_with_binding/invoke
              context.cljc:  149  tripod.context$leave_all$fn__13547/invoke
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  667  clojure.core/apply
                  core.clj: 1990  clojure.core/with-bindings*
                  core.clj: 1990  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
              context.cljc:  148  tripod.context$leave_all/invokeStatic
              context.cljc:  142  tripod.context$leave_all/invoke
              context.cljc:  199  tripod.context$execute/invokeStatic
              context.cljc:  198  tripod.context$execute/invoke
                 core.cljc:  113  martian.core$response_for/invokeStatic
                 core.cljc:  107  martian.core$response_for/invoke
                  core.clj:   36  wkok.openai-clojure.core/response-for
                  core.clj:   16  wkok.openai-clojure.core/response-for
                   api.clj:   84  wkok.openai-clojure.api/create-chat-completion
                   api.clj:   63  wkok.openai-clojure.api/create-chat-completion
                      REPL:  286  server.conversation/eval37315
                      REPL:  286  server.conversation/eval37315
             Compiler.java: 7194  clojure.lang.Compiler/eval
             Compiler.java: 7149  clojure.lang.Compiler/eval
                  core.clj: 3215  clojure.core/eval
                  core.clj: 3211  clojure.core/eval
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  667  clojure.core/apply
                  core.clj: 1990  clojure.core/with-bindings*
                  core.clj: 1990  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  437  clojure.main/repl/read-eval-print/fn
                  main.clj:  437  clojure.main/repl/read-eval-print
                  main.clj:  458  clojure.main/repl/fn
                  main.clj:  458  clojure.main/repl
                  main.clj:  368  clojure.main/repl
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  218  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  217  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java: 1583  java.lang.Thread/run
nsadeh commented 10 months ago

Based on my digging, one would probably want to follow Martian's convention of having separate functions or potentially even Martian specs for sync and async and use different ones depending on the request. Here's where Martian implements that: https://github.com/oliyh/martian/blob/732be670cd26d2a401b3830867589e33817e7fec/hato/src/martian/hato.clj

I tried to refactor my fork to be async-default just so I understand how I'd go about making it support both, but at the moment I can't get it to return a future, only a nil, by replacing the default interceptors with the async versions and modifying the sse.clj code to run the async version if it's not streaming. I'll try to do a bit more tomorrow.

Do you see a way to do it?

wkok commented 10 months ago

I got something going: PR https://github.com/wkok/openai-clojure/pull/53

Could you try it out / use it as reference in your implementation?