Binary executable created with Roswell cannot accept tcp connection

[ppsdatta]

I created a simple script with Roswell which runs the hello world code similar to README. When I run this script from command line without building, it runs fine and sends the html response to browser. But when I uses ros build to build the binary and run it instead, I get this error on Mac:

Error: The value # is not of the expected type MACPTR. While executing: WOO.LLSOCKET:ACCEPT, in process toplevel(3). Type :POP to abort, :R for a list of available restarts. Type :? for other options.

And on Ubuntu, I get this:

<ERROR> [09:50:45] woo.ev.tcp - Can't accept connection (Code: 22)

Can I see the code?

[ppsdatta]

w1.tar.gz

Here's the code (I have attached the file also)

#!/bin/sh
#|-*- mode:lisp -*-|#
#| <Put a one-line description here>
exec ros -Q -- $0 "$@"
|#
(progn ;;init forms
    (ql:quickload "woo"))

(defpackage :ros.script.w1.3677509307
(:use :cl))
(in-package :ros.script.w1.3677509307)

(defun main (&rest argv)
  (declare (ignorable argv))
  (woo:run
   (lambda (env)
     (declare (ignore env))
     '(200 (:content-type "text/plain") ("Hello, World")))) )

;;; vim: set ft=lisp lisp:

running it with ros w1.ros works, but building the binary with ros build w1.ros and running it produces the above error.

[knobo]

I don't think this is a woo bug. It is a known issue with ccl. Maybe you should ask on irc or some ccl forum on how to fix it. The problem is that the pointers to the foreign library is not stored in the built image. So the pointers has to be reloaded when the image is started. (I think)

[ppsdatta]

Yes I also think something like that may be the case. However this problem is occurring with SBCL too as I'm using that in both Linux and OS X.

[knobo]

I don't think you have MACPTR type on sbcl. I don't know anything about OS X though. But on Linux there are no MACPTRs on sbcl. So I don't thing you are using SBCL.

[ppsdatta]

Right, the first example error that I showed was from CCL. But I tried with SBCL as well on Mac and Linux and got the same second error which I pasted above. I think we can close this issue as it is not particular to Woo but rather how the binaries are being created on these platforms.

Get Outlook for Androidhttps://aka.ms/ghei36

On Tue, Jul 19, 2016 at 1:02 AM +0530, "Knut Olav Bøhmer" notifications@github.com<mailto:notifications@github.com> wrote:

I don't think you have MACPTR type on sbcl. I don't know anything about OS X though. But on Linux there are no MACPTRs on sbcl. So I don't thing you are using SBCL.

You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/fukamachi/woo/issues/51#issuecomment-233433559, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AR2L7FIhP7zWJmyHkO-zZtJWrXKN6jGtks5qW9TCgaJpZM4JNEQ0.

[knobo]

Could you show me the whole error for sbcl? Do you get an error when building with sbcl?

[ppsdatta]

Yes I got the error with sbcl but not during building. The error in all cases happens during running the built binary. Here are the roswell and default SBCL versions I am using on an Ubuntu 16.04 64 bit box.

$ ros version
roswell 0.0.6.64(c65d88b)
build with gcc (Ubuntu 5.3.1-14ubuntu2) 5.3.1 20160413
libcurl=7.47.0
Quicklisp=2016-02-22
Dist=2016-06-28

$ ros config
sbcl-bin.version=1.3.7
default.lisp=sbcl-bin

The build output is like below:

$ ros build w1.ros
To load "cffi-grovel":
  Load 1 ASDF system:
    cffi-grovel
; Loading "cffi-grovel"
.........
To load "woo":
  Load 1 ASDF system:
    woo
; Loading "woo"
....
[undoing binding stack and other enclosing state... done]
[saving current Lisp image into w1:
writing 4800 bytes from the read-only space at 0x20000000
writing 4240 bytes from the static space at 0x20100000
writing 68812800 bytes from the dynamic space at 0x1000000000
done]

And when I run the generated binary it waits for connections. On a new connection the following error is displayed.

$ ./w1
 <ERROR> [08:34:46] woo.ev.tcp - Can't accept connection (Code: 22)
 <ERROR> [08:34:46] woo.ev.tcp - Can't accept connection (Code: 22)
 <ERROR> [08:34:46] woo.ev.tcp - Can't accept connection (Code: 22)
 <ERROR> [08:34:46] woo.ev.tcp - Can't accept connection (Code: 22)
 <ERROR> [08:34:46] woo.ev.tcp - Can't accept connection (Code: 22)
 <ERROR> [08:34:51] woo.ev.tcp - Can't accept connection (Code: 22)

But running it in script mode with ros w1.ros does not cause this error and the data is sent to client.

[knobo]

I am not able to reproduce this. Could you trace it with strace, and paste the result?

EDIT: like this:

strace ./w1 > w1.strace.log 2>&1

Then do two requests with your browser, and upload the log.

[souravdatta]

Ran the trace and attaching file. I guess this the part when the error occurs:

strace.txt rt_sigaction(SIGINT, {0x7ffff22cabb0, ~[RTMIN RT_1], SA_RESTORER|SA_RESTART, 0x7ffff79c73d0}, NULL, 8) = 0 rt_sigaction(SIGQUIT, {0x7ffff22cabb0, ~[RTMIN RT_1], SA_RESTORER|SA_RESTART, 0x7ffff79c73d0}, NULL, 8) = 0 socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 7 fcntl(7, F_GETFL) = 0x2 (flags O_RDWR) setsockopt(7, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0 bind(7, {sa_family=AF_INET, sin_port=htons(5000), sin_addr=inet_addr("0.0.0.0")}, 16) = 0 listen(7, 128) = 0 epoll_ctl(5, EPOLL_CTL_ADD, 6, {EPOLLIN, {u32=6, u64=4294967302}}) = 0 epoll_ctl(5, EPOLL_CTL_ADD, 7, {EPOLLIN, {u32=7, u64=4294967303}}) = 0 epoll_wait(5, [{EPOLLIN, {u32=7, u64=4294967303}}], 64, 59743) = 1 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0 accept4(7, 0x641240, 0x641530, SOCK_CLOEXEC|SOCK_NONBLOCK) = -1 EINVAL (Invalid argument) --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_ACCERR, si_addr=0x10030c6eb8} --- mprotect(0x10030c0000, 32768, PROT_READ|PROT_WRITE|PROT_EXEC) = 0 rt_sigreturn({mask=[]}) = 68770623167 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_ACCERR, si_addr=0x10005c1d48} --- mprotect(0x10005c0000, 32768, PROT_READ|PROT_WRITE|PROT_EXEC) = 0 rt_sigreturn({mask=[]}) = 6036800 write(1, " [14:17:46] woo.ev.tcp -"..., 68 [14:17:46] woo.ev.tcp - Can't accept connection (Code: 22) ) = 68 epoll_wait(5, [{EPOLLIN, {u32=7, u64=4294967303}}], 64, 59743) = 1 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0 accept4(7, 0x641240, 0x641530, SOCK_CLOEXEC|SOCK_NONBLOCK) = -1 EINVAL (Invalid argument) write(1, " [14:18:03] woo.ev.tcp -"..., 68 [14:18:03] woo.ev.tcp - Can't accept connection (Code: 22) ) = 68 epoll_wait(5,

[knobo]

This looks like a Ubuntu 16.04 bug. Because it works on older Ubuntu versions, but not 16.04. I installed a virtual server with ubuntu 16.04 and saved an image, and it did not work. If I copied the image from my Ubuntu 15.10 it works fine.

[knobo]

Same problem on Archlinux

[souravdatta]

Ok, what's interesting is that it works fine on 16.04 if we don't make the binary and execute as a script. So probably something is wrong in the build system.

[knobo]

When I evaluate this line in a saved image: https://github.com/fukamachi/woo/blob/master/src/ev/tcp.lisp#L140 Then the server works again. So when I add:

(setf (cffi:mem-aref woo.ev.tcp::*dummy-socklen* 'wsock:socklen-t) (cffi:foreign-type-size '(:struct wsock:sockaddr-in)))

to the main function in w.ros, then it works. It is not a solution, but a step in the debugging.

[knobo]

stassats on #sbcl explains that sbcl does not save foreign memory, and that this stuff has to be added to init-hooks. So we'll have to make some init functions.

[knobo]

My fork works for now: https://github.com/knobo/woo

And I will think about if this patch is good enough or if some of the memory allocation could be moved in to a function that is called when woo is started, and if the memory in *dummy-sockaddr* should be freed before saving the image. Maybe @fukamachi has a good ide on how this is best fixed. EDIT: if sbcl does not store the foreign memory, then there is no need to free it begore saving.

[knobo]

Would be great if someone could test this on OS X also.

[svetlyak40wt]

I've encontered this problem too. @knobo will try your patch yesterday.

[fukamachi]

Merged #52. Sorry for the delay and thank you @knobo!