search

metosin / compojure-api

Sweet web apis with Compojure & Swagger
http://metosin.github.io/compojure-api/doc/
Eclipse Public License 1.0
1.12k stars 149 forks source link

Can't Figure Out Middleware with Async #389

Closed micahasmith closed 5 years ago

micahasmith commented 6 years ago

Library Version(s)

[ring "1.7.0-RC1"]
[metosin/compojure-api "1.1.11"]

Problem

I have an api that has async? true set in its config. For all of my handlers that are sync, and don't need the async 3-arity stuff, everything works well.

But for my async 3-arity handlers, the middleware isn't working. Here's the code and let me explain:

Middleware Handler

(defn auth-middleware [handler]
  (info ["handler" handler ])
  (fn
    ([request]
     (info ["sync middle"])
     (let [{:keys [error request] :as auth} (-auth-middleware request)]
       (info ["auth" auth] )
       (condp = error
         true (r/bad-request {:error  true
                              :errors [{:message "invalid recipeId/partnerBillingKey"}]})
         false (handler request))))
    ([request respond raise]
     (info ["async middle"])
     (let [{:keys [error request]} (auth-middleware request)]
       (condp = error
         true (handler request respond #(raise (r/bad-request {
                                                               :error  true
                                                               :errors [{:message "invalid recipeId/partnerBillingKey"}]})))
         false (handler request #(respond request) raise))))))

So you can see from above, i have the 3-arity and single arity fns. Surprisingly, the 3-arity middleware fn doesnt get called on my async functions. Instead, it calls the single arity one.

And then, it throws an error on (handler request) like so:

18-08-22 20:45:28 lifted ERROR [gooten-preview-api.www.app:101] - [#error {
 :cause "Wrong number of args (1) passed to: app/fn--40304/fn--40321/fn--40323"
 :via
 [{:type clojure.lang.ArityException
   :message "Wrong number of args (1) passed to: app/fn--40304/fn--40321/fn--40323"
   :at [clojure.lang.AFn throwArity "AFn.java" 429]}]
 :trace
 [[clojure.lang.AFn throwArity "AFn.java" 429]
  [clojure.lang.AFn invoke "AFn.java" 32]
  [compojure.response$eval1959$fn__1960 invoke "response.clj" 47]
  [compojure.response$eval1881$fn__1882$G__1872__1889 invoke "response.clj" 7]
  [compojure.core$wrap_response$fn__3808 invoke "core.clj" 158]
  [compojure.core$pre_init$fn__3907 invoke "core.clj" 328]
  [compojure.api.coerce$body_coercer_middleware$fn__25286 invoke "coerce.clj" 51]
  [compojure.core$pre_init$fn__3909$fn__3912 invoke "core.clj" 335]
  [compojure.core$wrap_route_middleware$fn__3792 invoke "core.clj" 127]
  [compojure.core$wrap_route_info$fn__3797 invoke "core.clj" 137]
  [compojure.core$wrap_route_matches$fn__3801 invoke "core.clj" 146]
  [compojure.core$wrap_routes$fn__3919 invoke "core.clj" 348]
  [compojure.api.routes.Route invoke "routes.clj" 74]
  [compojure.core$routing$fn__3816 invoke "core.clj" 185]
  [clojure.core$some invokeStatic "core.clj" 2693]
  [clojure.core$some invoke "core.clj" 2684]
  [compojure.core$routing invokeStatic "core.clj" 185]
  [compojure.core$routing doInvoke "core.clj" 182]
  [clojure.lang.RestFn applyTo "RestFn.java" 139]
  [clojure.core$apply invokeStatic "core.clj" 659]
  [clojure.core$apply invoke "core.clj" 652]
  [compojure.core$routes$fn__3820 invoke "core.clj" 192]
  [gooten_preview_api.www.app$auth_middleware$fn__40117 invoke "app.clj" 47]
  [compojure.core$routing$fn__3816 invoke "core.clj" 185]
  [clojure.core$some invokeStatic "core.clj" 2693]
  [clojure.core$some invoke "core.clj" 2684]
  [compojure.core$routing invokeStatic "core.clj" 185]
  [compojure.core$routing doInvoke "core.clj" 182]
  [clojure.lang.RestFn applyTo "RestFn.java" 139]
  [clojure.core$apply invokeStatic "core.clj" 659]
  [clojure.core$apply invoke "core.clj" 652]
  [compojure.core$routes$fn__3820 invoke "core.clj" 192]
  [compojure.core$make_context$handler__3888 invoke "core.clj" 285]
  [compojure.core$make_context$fn__3890 invoke "core.clj" 293]
  [compojure.api.routes.Route invoke "routes.clj" 74]
  [compojure.api.core$handle$fn__25497 invoke "core.clj" 8]
  [clojure.core$some invokeStatic "core.clj" 2693]
  [clojure.core$some invoke "core.clj" 2684]
  [compojure.api.core$handle invokeStatic "core.clj" 8]
  [compojure.api.core$handle invoke "core.clj" 7]
  [clojure.core$partial$fn__5561 invoke "core.clj" 2616]
  [compojure.api.routes.Route invoke "routes.clj" 74]
  [ring.swagger.middleware$wrap_swagger_data$fn__24659 invoke "middleware.clj" 35]
  [ring.middleware.http_response$wrap_http_response$fn__20572 invoke "http_response.clj" 19]
  [ring.swagger.middleware$wrap_swagger_data$fn__24659 invoke "middleware.clj" 35]
  [compojure.api.middleware$wrap_options$fn__24721 invoke "middleware.clj" 74]
  [ring.middleware.format_params$wrap_format_params$fn__20291 invoke "format_params.clj" 117]
  [ring.middleware.format_params$wrap_format_params$fn__20291 invoke "format_params.clj" 119]
  [ring.middleware.format_params$wrap_format_params$fn__20291 invoke "format_params.clj" 119]
  [ring.middleware.format_params$wrap_format_params$fn__20291 invoke "format_params.clj" 119]
  [ring.middleware.format_params$wrap_format_params$fn__20291 invoke "format_params.clj" 119]
  [compojure.api.middleware$wrap_exceptions$fn__24711 invoke "middleware.clj" 43]
  [ring.middleware.format_response$wrap_format_response$fn__20468 invoke "format_response.clj" 194]
  [ring.middleware.keyword_params$wrap_keyword_params$fn__20618 invoke "keyword_params.clj" 53]
  [ring.middleware.nested_params$wrap_nested_params$fn__20676 invoke "nested_params.clj" 89]
  [ring.middleware.params$wrap_params$fn__4048 invoke "params.clj" 67]
  [compojure.api.middleware$wrap_options$fn__24721 invoke "middleware.clj" 74]
  [compojure.api.routes.Route invoke "routes.clj" 74]
  [clojure.lang.Var invoke "Var.java" 381]
  [ring.middleware.reload$wrap_reload$fn__1828 invoke "reload.clj" 39]
  [ring.middleware.stacktrace$wrap_stacktrace_log$fn__1210 invoke "stacktrace.clj" 26]
  [ring.middleware.stacktrace$wrap_stacktrace_web$fn__1276 invoke "stacktrace.clj" 96]
  [ring.adapter.jetty$proxy_handler$fn__486 invoke "jetty.clj" 26]
  [ring.adapter.jetty.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a handle nil -1]
  [org.eclipse.jetty.server.handler.HandlerWrapper handle "HandlerWrapper.java" 97]
  [org.eclipse.jetty.server.Server handle "Server.java" 499]
  [org.eclipse.jetty.server.HttpChannel handle "HttpChannel.java" 311]
  [org.eclipse.jetty.server.HttpConnection onFillable "HttpConnection.java" 258]
  [org.eclipse.jetty.io.AbstractConnection$2 run "AbstractConnection.java" 544]
  [org.eclipse.jetty.util.thread.QueuedThreadPool runJob "QueuedThreadPool.java" 635]
  [org.eclipse.jetty.util.thread.QueuedThreadPool$3 run "QueuedThreadPool.java" 555]

Endpoint Code

(POST "/generate/" []
        :tags ["generate"]
        :summary "Create a product preview"
        :return api-schemas/GenerateResponse
        :query-params [recipeId :- s/Str
                       partnerBillingKey :- s/Str
                       ]
        :body [request api-schemas/GenerateRequest]

        (fn [request respond raise]
          (info ["body " request respond raise])
          (go-safe
            (-> (fn<? ppml-runner/generate! request)
                (async-handle respond raise)))

          ))
seancorfield commented 6 years ago

@micahasmith You seem to be calling (-auth-middleware request) in the 1-arity arm but (auth-middleware request) in the 3-arity arm, which is just going to be a recursive call into the 1-arity arm?

micahasmith commented 6 years ago

@seancorfield thx. thats definitely a bug, but not involved with this error unfortunately. from the logs i can tell it never hits the async middleware in the first place:

8-08-22 21:43:21 lifted INFO [gooten-preview-api.www.app:37] - ["handler" #object[compojure.core$routes$fn__3820 0x46871840 "compojure.core$routes$fn__3820@46871840"]]
18-08-22 21:43:21 lifted INFO [gooten-preview-api.www.app:40] - ["sync middle"]
micahasmith commented 6 years ago

To make this even easier to talk about and diagnose, i've made a bare minimum version of the problem using the lein new compojure-api template

handler.clj

(ns asyncy.handler
  (:require [compojure.api.sweet :refer :all]
            [ring.util.http-response :refer :all]
            [schema.core :as s]
            [clojure.core.async :as async]))

(s/defschema Pizza
  {:name                         s/Str
   (s/optional-key :description) s/Str
   :size                         (s/enum :L :M :S)
   :origin                       {:country (s/enum :FI :PO)
                                  :city    s/Str}})

(def app
  (api
    {:async?     true
     :swagger
                 {:ui   "/"
                  :spec "/swagger.json"
                  :data {:info {:title       "Asyncy"
                                :description "Compojure Api example"}
                         :tags [{:name "api", :description "some apis"}]}}}

    (context "/api" []
      :tags ["api"]
      :middleware [(fn [handler]
                     ;; this sync-style handler always handles both sync and async requests
                     (fn ([request]
                          (clojure.pprint/pprint request)
                          (if (some-> request :params :x (= "1"))
                            (bad-request {:error true :message "one is the loneliest number that you'll ever do"})
                            ;; this throws an error on async requests
                            (handler request)))

                       ;; this async 3-arity handler is never called
                       ([request b c]
                        (clojure.pprint/pprint ["never called, doesnt matter." request b c]))
                       ))]

      (GET "/plus" []
        :return {:result Long}
        :query-params [x :- Long, y :- Long]
        :summary "adds two numbers together"
        (ok {:result (+ x y)}))

      (GET "/plus-async" []
        :return {:result Long}
        :query-params [x :- Long, y :- Long]
        :summary "adds two numbers together"
        (fn [request respond raise]
          (respond (ok {:result (+ x y)}))))
      )))

(handler request) in the middleware is what throws the error, for async requests only:

ERROR Wrong number of args (1) passed to: handler/fn--26289/fn--26301/fn--26303
clojure.lang.ArityException: Wrong number of args (1) passed to: handler/fn--26289/fn--26301/fn--26303
        at clojure.lang.AFn.throwArity(AFn.java:429)
        at clojure.lang.AFn.invoke(AFn.java:32)
        at compojure.response$eval1960$fn__1961.invoke(response.clj:47)
        at compojure.response$eval1882$fn__1883$G__1873__1890.invoke(response.clj:7)
        at compojure.core$wrap_response$fn__3839.invoke(core.clj:158)
        at compojure.core$pre_init$fn__3938.invoke(core.clj:328)
        at compojure.api.coerce$body_coercer_middleware$fn__14642.invoke(coerce.clj:51)
        at compojure.core$pre_init$fn__3940$fn__3943.invoke(core.clj:335)
        at compojure.core$wrap_route_middleware$fn__3823.invoke(core.clj:127)
        at compojure.core$wrap_route_info$fn__3828.invoke(core.clj:137)
        at compojure.core$wrap_route_matches$fn__3832.invoke(core.clj:146)
        at compojure.core$wrap_routes$fn__3950.invoke(core.clj:348)
        at compojure.api.routes.Route.invoke(routes.clj:74)
        at compojure.core$routing$fn__3847.invoke(core.clj:185)
        at clojure.core$some.invokeStatic(core.clj:2693)
        at clojure.core$some.invoke(core.clj:2684)
        at compojure.core$routing.invokeStatic(core.clj:185)
        at compojure.core$routing.doInvoke(core.clj:182)
        at clojure.lang.RestFn.applyTo(RestFn.java:139)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$apply.invoke(core.clj:652)
        at compojure.core$routes$fn__3851.invoke(core.clj:192)
        at asyncy.handler$fn__26289$fn__26290$fn__26291.invoke(handler.clj:32)
        at compojure.core$routing$fn__3847.invoke(core.clj:185)
        at clojure.core$some.invokeStatic(core.clj:2693)
        at clojure.core$some.invoke(core.clj:2684)
        at compojure.core$routing.invokeStatic(core.clj:185)
        at compojure.core$routing.doInvoke(core.clj:182)
        at clojure.lang.RestFn.applyTo(RestFn.java:139)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$apply.invoke(core.clj:652)
        at compojure.core$routes$fn__3851.invoke(core.clj:192)
        at compojure.core$make_context$handler__3919.invoke(core.clj:285)
        at compojure.core$make_context$fn__3921.invoke(core.clj:293)
        at compojure.api.routes.Route.invoke(routes.clj:74)
        at compojure.api.core$handle$fn__14853.invoke(core.clj:8)
        at clojure.core$some.invokeStatic(core.clj:2693)
        at clojure.core$some.invoke(core.clj:2684)
        at compojure.api.core$handle.invokeStatic(core.clj:8)
        at compojure.api.core$handle.invoke(core.clj:7)
        at clojure.core$partial$fn__5561.invoke(core.clj:2616)
        at compojure.api.routes.Route.invoke(routes.clj:74)
        at ring.swagger.middleware$wrap_swagger_data$fn__14015.invoke(middleware.clj:35)
        at ring.middleware.http_response$wrap_http_response$fn__8034.invoke(http_response.clj:19)
        at ring.swagger.middleware$wrap_swagger_data$fn__14015.invoke(middleware.clj:35)
        at compojure.api.middleware$wrap_options$fn__14077.invoke(middleware.clj:74)
        at ring.middleware.format_params$wrap_format_params$fn__7105.invoke(format_params.clj:119)
        at ring.middleware.format_params$wrap_format_params$fn__7105.invoke(format_params.clj:119)
        at ring.middleware.format_params$wrap_format_params$fn__7105.invoke(format_params.clj:119)
        at ring.middleware.format_params$wrap_format_params$fn__7105.invoke(format_params.clj:119)
        at ring.middleware.format_params$wrap_format_params$fn__7105.invoke(format_params.clj:119)
        at compojure.api.middleware$wrap_exceptions$fn__14067.invoke(middleware.clj:43)
        at ring.middleware.format_response$wrap_format_response$fn__7930.invoke(format_response.clj:194)
        at ring.middleware.keyword_params$wrap_keyword_params$fn__8076.invoke(keyword_params.clj:36)
        at ring.middleware.nested_params$wrap_nested_params$fn__8134.invoke(nested_params.clj:89)
        at ring.middleware.params$wrap_params$fn__4079.invoke(params.clj:67)
        at compojure.api.middleware$wrap_options$fn__14077.invoke(middleware.clj:74)
        at compojure.api.routes.Route.invoke(routes.clj:74)
        at clojure.lang.Var.invoke(Var.java:381)
        at ring.middleware.reload$wrap_reload$fn__1829.invoke(reload.clj:39)
        at ring.middleware.stacktrace$wrap_stacktrace_log$fn__1211.invoke(stacktrace.clj:26)
        at ring.middleware.stacktrace$wrap_stacktrace_web$fn__1277.invoke(stacktrace.clj:96)
        at ring.adapter.jetty$proxy_handler$fn__487.invoke(jetty.clj:25)
        at ring.adapter.jetty.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a.handle(Unknown Source)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:499)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258)
        at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:748)
ikitommi commented 6 years ago

Hi. The async support is a feature in 2.* branch. There is a async guide over here: https://github.com/metosin/compojure-api/wiki/Async

micahasmith commented 6 years ago

@ikitommi a single example of how to do middleware would be very helpful 🙏

micahasmith commented 6 years ago

ok. so with alpha 23 we're still no good on this problem.

   (def app
  (api
    {:async? true
     :swagger
             {:ui   "/"
              :spec "/swagger.json"
              :data {:info {:title       "Asyncy"
                            :description "Compojure Api example"}
                     :tags [{:name "api", :description "some apis"}]}}}
    (context "/api" []
      :tags ["api"]
      ;; this works
      (GET "/plus" []
        :return {:result Long}
        :query-params [x :- Long, y :- Long]
        :summary "adds two numbers together"
        :middleware [(fn [handler]
                       ;; this sync-style handler always handles both sync and async requests
                       (fn [request]
                         (clojure.pprint/pprint request)
                         (if (some-> request :params :x (= "1"))
                           (bad-request {:error true :message "one is the loneliest number that you'll ever do"})
                           ;; this throws an error on async requests
                           (handler request))
                         ))]
        (ok {:result (+ x y)}))

      ;; this doesnt work
      (GET "/plus-async" []
        :return {:result Long}
        :query-params [x :- Long, y :- Long]
        :summary "adds two numbers together"
        :middleware [(fn [handler]
                       (fn [request b c]
                         (clojure.pprint/pprint [request b c])
                         (handler request)))]
        (fn [request respond raise]
          (prn [request respond raise])
          (respond (ok {:result (+ x y)}))))
      )))

(def server (run-jetty app {:port 3002}))

Can't get either 1 or 3 arity to work.

ikitommi commented 6 years ago

The :async? true needs to be set to jetty, which is currently the only ring-adapter supporting it. Updated the example project to latest deps: https://github.com/metosin/compojure-api/tree/master/examples/async

ikitommi commented 6 years ago

the api (and all the default middleware) always support both 1 and 3 arity.

ikitommi commented 5 years ago

closing this now.

atifh commented 4 years ago

@ikitommi a middleware example would be very helpful. /cc @micahasmith

ikitommi commented 4 years ago

Something like: https://github.com/metosin/compojure-api/blob/master/src/compojure/api/middleware.clj#L185-L192