Cannot send JSON

[jcoleman-techempower]

I do not have a lot of experience with Common Lisp, but I would like to learn it a bit better. I am having trouble with :content-type "application/json". This is running with Docker, using Roswell. Here is the stack trace:

woo: Unhandled FAST-HTTP.ERROR:CB-MESSAGE-COMPLETE in thread #<SB-THREAD:THREAD
woo:                                                           "main thread" RUNNING
woo:                                                            {1002D7FEA3}>:
woo:   Callback Error: the message-complete callback failed
woo:   The value TO-JSON is not of type STRING.
woo: Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {1002D7FEA3}>
woo: 0: (TRIVIAL-UTF-8:UTF-8-BYTE-LENGTH TO-JSON) [tl,external]
woo: 1: (TRIVIAL-UTF-8:STRING-TO-UTF-8-BYTES #<unavailable argument> :NULL-TERMINATE #<unavailable argument>)
woo: 2: (WOO::HANDLE-NORMAL-RESPONSE #<unavailable argument> #<unavailable argument> #<unavailable argument>)
woo: 3: (WOO::HANDLE-RESPONSE #S(FAST-HTTP.HTTP:HTTP-REQUEST :METHOD :GET :MAJOR-VERSION 1 :MINOR-VERSION 1 :STATUS 0 :CONTENT-LENGTH NIL :CHUNKED-P NIL :UPGRADE-P NIL :HEADERS #<HASH-TABLE :TEST EQUAL :COUNT 3 {1003068DD3}> :HEADER-READ 0 :MARK 83 :STATE 3 :RESOURCE "/json") #S(WOO.EV.SOCKET:SOCKET :WATCHERS #(#.(SB-SYS:INT-SAP #X006502E0) #.(SB-SYS:INT-SAP #X00650490) #.(SB-SYS:INT-SAP #X006504D0)) :LAST-ACTIVITY 1.5445727350415163d9 :FD 9 :REMOTE-ADDR "172.18.0.4" :REMOTE-PORT 34485 :DATA #<CLOSURE (LAMBDA (FAST-HTTP::DATA &KEY (FAST-HTTP::START 0) FAST-HTTP::END) :IN FAST-HTTP:MAKE-PARSER) {100306899B}> :TCP-READ-CB WOO.EV.TCP::TCP-READ-CB :READ-CB #<FUNCTION WOO::READ-CB> :WRITE-CB NIL :OPEN-P T :BUFFER #S(FAST-IO::OUTPUT-BUFFER :VECTOR #(58 32 87 111 111 13 10 84 114 97 110 115 ...) :FILL 37 :LEN 153 :QUEUE #1=(#(72 84 84 80 47 49 46 49 32 50 48 48 ...) #(10 68 97 116 101 58 32 84 117 101 44 32 ...) #(71 77 84 13 10 67 111 110 110 101 99 116 ...)) :LAST #1# :OUTPUT NIL)) (200 (:CONTENT-TYPE "application/json" :SERVER "Woo") (TO-JSON (QUOTE (:MESSAGE "Hello, World!")))))
woo: 4: ((LAMBDA (FAST-HTTP.HTTP:HTTP) :IN FAST-HTTP:MAKE-PARSER) #<unavailable argument>)
woo: 5: (FAST-HTTP.PARSER::PARSE-BODY #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument>)
woo: 6: (FAST-HTTP.PARSER:PARSE-REQUEST #<unavailable argument> #<unavailable argument> #<unavailable argument> :START #<unavailable argument> :END #<unavailable argument>)
woo: 7: ((LAMBDA (FAST-HTTP::DATA &KEY (FAST-HTTP::START 0) FAST-HTTP::END) :IN FAST-HTTP:MAKE-PARSER) #<unavailable argument> :START #<unavailable argument> :END #<unavailable argument>)
woo: 8: (WOO::READ-CB #S(WOO.EV.SOCKET:SOCKET :WATCHERS #(#.(SB-SYS:INT-SAP #X006502E0) #.(SB-SYS:INT-SAP #X00650490) #.(SB-SYS:INT-SAP #X006504D0)) :LAST-ACTIVITY 1.5445727350415163d9 :FD 9 :REMOTE-ADDR "172.18.0.4" :REMOTE-PORT 34485 :DATA #<CLOSURE(LAMBDA (FAST-HTTP::DATA &KEY (FAST-HTTP::START 0) FAST-HTTP::END) :IN FAST-HTTP:MAKE-PARSER) {100306899B}> :TCP-READ-CB WOO.EV.TCP::TCP-READ-CB :READ-CB #<FUNCTION WOO::READ-CB> :WRITE-CB NIL :OPEN-P T :BUFFER #S(FAST-IO::OUTPUT-BUFFER :VECTOR #(58 32 87 111 111 13 10 84 114 97 110 115 ...) :FILL 37 :LEN 153 :QUEUE #1=(#(72 84 84 80 47 49 46 49 32 50 48 48 ...) #(10 68 97 116 101 58 32 84 117 101 44 32 ...) #(71 77 84 13 10 67 111 110 110 101 99 116 ...)) :LAST #1# :OUTPUT NIL)) #(71 69 84 32 47 106 115 111 110 32 72 84 ...) :START 0 :END 83)
woo: 9: (WOO.EV.TCP::TCP-READ-CB #<unavailable argument> #.(SB-SYS:INT-SAP #X006502E0) #<unavailable argument>)
woo: 10: ((LAMBDA (SB-ALIEN::ARGS-POINTER SB-ALIEN::RESULT-POINTER FUNCTION) :IN "/root/.cache/common-lisp/sbcl-1.2.12-linux-x64/1.2.12/root/.roswell/impls/ALL/ALL/quicklisp/dists/quicklisp/software/woo-20150608-git/src/ev/socket.fasl") 7036863545102470368635451020 #<FUNCTION (LAMBDA (WOO.EV.TCP::EVLOOP WOO.EV.TCP::WATCHER WOO.EV.TCP::EVENTS) :IN "/root/.roswell/impls/ALL/ALL/quicklisp/dists/quicklisp/software/woo-20150608-git/src/ev/tcp.lisp") {10067ABD4B}>)
woo: 11: ("foreign function: call_into_lisp")
woo: 12: ("foreign function: funcall3")
woo: 13: ("foreign function: callback_wrapper_trampoline")
woo: 14: ("foreign function: #x20100D1C")
woo: 15: ((FLET WOO::START-SERVER-MULTI :IN WOO:RUN))
woo: 16: (WOO:RUN #<FUNCTION (LAMBDA (ENV) :IN MAIN) {100306525B}> :DEBUG NIL :PORT 8080 :ADDRESS "0.0.0.0" :BACKLOG NIL :FD NIL :WORKER-NUM 4)
woo: 17: (SB-INT:SIMPLE-EVAL-IN-LEXENV (APPLY (QUOTE MAIN) ROS:*ARGV*) #<NULL-LEXENV>)
woo: 18: (EVAL-TLF (APPLY (QUOTE MAIN) ROS:*ARGV*) NIL #<NULL-LEXENV>)
woo: 19: ((LABELS SB-FASL::EVAL-FORM :IN SB-INT:LOAD-AS-SOURCE) (APPLY (QUOTE MAIN) ROS:*ARGV*) NIL)
woo: 20: (SB-INT:LOAD-AS-SOURCE #<CONCATENATED-STREAM :STREAMS NIL {1005EB81B3}> :VERBOSE NIL :PRINT NIL :CONTEXT "loading")
woo: 21: ((FLET SB-FASL::LOAD-STREAM :IN LOAD) #<CONCATENATED-STREAM :STREAMS NIL {1005EB81B3}> NIL)
woo: 22: (LOAD #<CONCATENATED-STREAM :STREAMS NIL {1005EB81B3}> :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST T :EXTERNAL-FORMAT:DEFAULT)
woo: 23: (ROS:SCRIPT :SCRIPT "./woo.ros" "--worker" "4" "--port" "8080")
woo: 24: (ROS:RUN ((:SCRIPT "./woo.ros" "--worker" "4" "--port" "8080") (:QUIT NIL)))
woo: 25: (SB-INT:SIMPLE-EVAL-IN-LEXENV (ROS:RUN (QUOTE ((:SCRIPT "./woo.ros" "--worker" "4" "--port" "8080") (:QUIT NIL)))) #<NULL-LEXENV>)
woo: 26: (EVAL (ROS:RUN (QUOTE ((:SCRIPT "./woo.ros" "--worker" "4" "--port" "8080") (:QUIT NIL)))))
woo: 27: (SB-IMPL::PROCESS-EVAL/LOAD-OPTIONS ((:EVAL . "(progn #-ros.init(cl:load \"/usr/local/share/common-lisp/source/roswell/init.lisp\"))") (:EVAL . "(ros:quicklisp)") (:EVAL . "(ros:run '((:script \"./woo.ros\"\"--worker\"\"4\"\"--port\"\"8080\")(:quit ())))")))
woo: 28: (SB-IMPL::TOPLEVEL-INIT)
woo: 29: ((FLET #:WITHOUT-INTERRUPTS-BODY-83 :IN SAVE-LISP-AND-DIE))
woo: 30: ((LABELS SB-IMPL::RESTART-LISP :IN SAVE-LISP-AND-DIE))
woo: unhandled condition in --disable-debugger mode, quitting

Source

[libre-man]

You return a quoted list (here). This means that the content is not evaluated, so the value is never converted to json. You should change this line to something like (list 200 '(:content-type "application/json" :server "Woo") (jonathan:to-json '(:message "Hello, World!"))) (not tested).

As this is for a benchmark: This means that the list is consed every time the function is called, and the conversion is also done every time. If you want to compare the speed of woo you need to make sure that this overhead isn't too large.

[jcoleman-techempower]

@libre-man Could you point me in the direction to find out what ', the back tick, ,, and @ mean in CL?

Edit: Also, why am I supposed to return a list when I am trying to return a JSON object? I'm trying to understand this a bit better.

[jcoleman-techempower]

That was actually a super helpful answer. I was trying to figure out how CL returns specific types, but I couldn't really understand it at a first glance.

[libre-man]

There is no JSON object, you probably want to return a JSON response. A response in woo is a normal list consisting of three items: the first item is the response code, the second is an alist that represent the headers and the optional third and final item is the response body.

You can learn more about the quote macro, backtick, comma and at operator in the practical Common Lisp book, especially this chapter explains the quote macro (') and this chapter explains the others.

Edit: The quote and backtick are probably better explained in this chapter.

[jcoleman-techempower]

@libre-man I cannot thank you enough for your insight on this. Thank you. I'll give them both a read. Going back to the original question, it makes perfect sense that a Woo response is a list with 3 items. Making the changes you had suggested above returns now the following:

woo: Unhandled FAST-HTTP.ERROR:CB-MESSAGE-COMPLETE in thread #<SB-THREAD:THREAD
woo:                                                           "main thread" RUNNING
woo:                                                            {1002D7FEA3}>:
woo:   Callback Error: the message-complete callback failed
woo:   "{\"MESSAGE\":\"Hello, World!\"}" fell through ETYPECASE expression.
woo:   Wanted one of (NULL PATHNAME LIST (VECTOR (UNSIGNED-BYTE 8))).

Is this because MESSAGE is returned instead of message?

[libre-man]

Sorry I made a mistake in the snippet. The third argument should be a list of strings, so (list 200 '(:content-type "application/json" :server "Woo") (list (jonathan:to-json '(:message "Hello, World!")))) should work.

You could rewrite to this with ` to make to code look a bit nicer.

[jcoleman-techempower]

You would be correct. Thank you for all your help. I am curious how the backtick will make it cleaner, so I'll take a deep dive into the syntax of the language. I appreciate the great explanations