chain of pexecs creating channels: failure

[hraban]

(This message was actually intended for chanl's mailing list, but I have no idea of where to find that. For lack of a better place to post it, here it is.)

Hi,

I am trying to get some chanl equivalents to simple Google Go (commonlisp + chanl's imperative equivalent) programs going, but I am having some trouble. Using the test program included below, one can easily make a chain of millions of coroutines that pass along the same channel to the next one, until one of them finally sends something on it, back to the root function.

However, problems arise when you try to have every coroutine create a new channel to its successor. A sort of "linked list" of coroutines, if you will. I can not even get to 2 coroutines: the OS throws a "mmap: Cannot allocate memory" error and chanl returns an error. Full output below.

Trying out the program below on sbcl + ubuntu 10.10 works fine for (channel-reuse), but fails for (channel-chain). Does anybody have some ideas?

(ql:quickload "chanl")
(rename-package :chanl :chanl '(:c)) ;; RP hack

(defvar *limit* 1000)

(in-package :c)
(defun unzip-alist (alist) (values (mapcar #'car alist) (mapcar #'cdr alist)))
(in-package :cl-user)

(defun channel-chain (i reschan)
  (if (eql i *limit*)
    (c:send reschan i)
    (let ((argchan (make-instance 'c:channel)))
      (c:pexec () (channel-chain (1+ i) argchan))
      (c:send reschan (c:recv argchan)))))

(defun channel-reuse (i c)
  (if (eql i *limit*)
    (c:send c i)
    (c:pexec () (channel-reuse (1+ i) c))))

(defun call-chan-func (f)
  (format T "Calling ~S~%" f)
  (let ((c (make-instance 'c:channel)))
    (time
      (progn
        (c:pexec () (funcall f 0 c))
        (c:recv c)))))

(defun call-chan-funcs (&rest fs)
  (mapc #'call-chan-func fs))

(defun main ()
  (call-chan-funcs #'channel-reuse #'channel-chain))

Output of (main):

* (main)
Calling #<FUNCTION CHANNEL-REUSE>
Evaluation took:
  0.055 seconds of real time
  0.060004 seconds of total run time (0.056004 user, 0.004000 system)
  109.09% CPU
  25 lambdas converted
  166,717,028 processor cycles
  1,363,504 bytes consed

Calling #<FUNCTION CHANNEL-CHAIN>
mmap: Cannot allocate memory

debugger invoked on a SIMPLE-ERROR in thread #<THREAD
                                               "ChanL Thread Pool Worker" RUNNING
                                               {BC5B1D9}>:
  Can't create a new thread

[zkat]

Works for me on SBCL 1.0.49 on Linux 2.6.39 x86-64:

CL-USER> (main)
Calling #<FUNCTION CHANNEL-REUSE>
Evaluation took:
  0.059 seconds of real time
  0.083328 seconds of total run time (0.063330 user, 0.019998 system)
  140.68% CPU
  38 lambdas converted
  186,372,169 processor cycles
  4,615,904 bytes consed

Calling #<FUNCTION CHANNEL-CHAIN>
Evaluation took:
  61.706 seconds of real time
  61.092683 seconds of total run time (0.149990 user, 60.942693 system)
  99.01% CPU
  197,459,754,445 processor cycles
  6 page faults
  276,144 bytes consed

(#<FUNCTION CHANNEL-REUSE> #<FUNCTION CHANNEL-CHAIN>)

Although it bumped my SBCL's RES usage to 2.4G ;)

SBCL, unlike Go, isn't really meant to handle a high number of threads. SBCL's threading library creates native OS threads, which are much more heavyweight than Go's fibers. I'd just say "don't make so many threads and use thread pools to reuse the ones you created".

You do say that you couldn't even get to two threads -- what makes you say that? I assume you tried it with limit = 2? If so, I would classify this as an SBCL bug, since the thread itself is failing to allocate. If you still have the problem, I would recommend reporting it to the SBCL folks (and seeing if you can reproduce it in CCL, perhaps).

[adlai]

Tested on SBCL 1.0.57 on Archlinux 3.4.2 on a dual-core x86_64:

* (main)
Calling #<FUNCTION CHANNEL-REUSE>
Evaluation took:
  0.036 seconds of real time
  0.043331 seconds of total run time (0.039998 user, 0.003333 system)
  119.44% CPU
  38 lambdas converted
  86,412,483 processor cycles
  5,294,720 bytes consed

Calling #<FUNCTION CHANNEL-CHAIN>
Evaluation took:
  1.557 seconds of real time
  1.629893 seconds of total run time (0.186654 user, 1.443239 system)
  104.69% CPU
  3,725,523,063 processor cycles
  1,504 bytes consed
(#<FUNCTION CHANNEL-REUSE> #<FUNCTION CHANNEL-CHAIN>)

Also works on CCL 1.7, same system (although it takes obscenely long to do so):

? (main)
Calling #<Compiled-function CHANNEL-REUSE #x302000C7668F>
(PROGN (CHANL:PEXEC NIL (FUNCALL F 0 C)) (CHANL:RECV C)) took 53,942 microseconds (0.053942 seconds) to run 
                    with 2 available CPU cores.
During that period, 66,662 microseconds (0.066662 seconds) were spent in user mode
                    16,665 microseconds (0.016665 seconds) were spent in system mode
 8,944 bytes of memory allocated.
 41 minor page faults, 0 major page faults, 0 swaps.
Calling #<Compiled-function CHANNEL-CHAIN #x302000C467CF>
(PROGN (CHANL:PEXEC NIL (FUNCALL F 0 C)) (CHANL:RECV C)) took 30,874,309 microseconds (30.874308 seconds) to run 
                    with 2 available CPU cores.
During that period, 26,361,615 microseconds (26.361614 seconds) were spent in user mode
                    30,468,014 microseconds (30.468014 seconds) were spent in system mode
1,084,299 microseconds (1.084299 seconds) was spent in GC.
 272 bytes of memory allocated.
 11,968 minor page faults, 0 major page faults, 0 swaps.
(#<Compiled-function CHANNEL-REUSE #x3020008CF1DF> #<Compiled-function CHANNEL-CHAIN #x3020008CB10F>)

Unless you're still having problems with this, I'm closing the issue.

[hraban]

Hi, the system I used for this is gone. I tested it on a VM install with Debian and it works fine now. Pretty fast, too. I guess this can be filed as "can not reproduce". Greetings!