atlas-engineer / cl-electron

Lisp Interface to Electron.
BSD 3-Clause "New" or "Revised" License
16 stars 1 forks source link

`handle-callback` doesn't gracefully handle strings of length >= 2^16 #19

Closed aadcg closed 7 months ago

aadcg commented 7 months ago

Steps to reproduce:

(defun protocol-bug ()
  (setf electron:*interface* (make-instance 'electron:interface))
  (electron:launch)
  ;; See https://github.com/atlas-engineer/cl-electron/issues/15
  (sleep 1)
  (electron:handle-callback (make-instance 'electron:protocol :scheme-name "lisp")
                            (lambda (url)
                              (declare (ignorable url))
                              (make-string (expt 2 16) :initial-element #\a)))
  (electron:load-url (make-instance 'electron:browser-window) "lisp:hello"))
// console.log(jsonString);
{"callback":627,"request":"lisp:hello"}
// console.log(data.toString());
{"callback":627,"result":"aaaaaa(...)*newline*
// console.log(data.toString());
aaaaaaaaaaaaaaaaaaaaaaaaaa"}
jmercouris commented 7 months ago

Snippet that potentially proves Lisp is not to blame, and the socket failure is on the NodeJS side:

(iolib:with-open-socket (s :address-family :local
                                    :connect :passive
                                    :local-filename "/home/jmercouris/Downloads/test.sock")
           (setf (iolib/os:file-permissions "/home/jmercouris/Downloads/test.sock")
                 (set-difference (iolib/os:file-permissions "/home/jmercouris/Downloads/test.sock")
                                 '(:group-read :group-write :group-exec
                                   :other-read :other-write :other-exec)))
           (iolib:with-accept-connection (connection s)
             (loop for expr = (read-line connection nil)
                   until (null expr)
                   do (unless (uiop:emptyp expr)
                            (write-line (make-string (expt 2 16) :initial-element #\a) connection)
                            (finish-output connection)))))

Then run in a shell:

ncat -U /home/jmercouris/Downloads/test.sock

then, when ncat is interactive, type a letter, and press RET. You should now receive back some output from Lisp. I have tested it, and it does not appear to be truncated or have any newlines. However, you should verify this!

jmercouris commented 7 months ago

Indeed NodeJS is buffering our output for the stream to 2^16.

https://nodejs.org/api/stream.html#stream_buffering

you can see an example here. However, it doesn't /quite/ help us because it relies on the end event, and we don't want to utilize it. I have a potentially quite hacky solution I am exploring.

jmercouris commented 7 months ago

Hacky solution exists on branch: data-resolution-limit.

It is not ideal. I'll think about it more, but it is a solution.