segfault with many macros or many exported identifiers (or both)

[GoogleCodeExporter]

What steps will reproduce the problem?
1. At the REPL import the library (foreign databases sqlite sizeof) from
[1], making it possible for Mosh to find the prerequisite libraries.

[1]
<http://github.com/marcomaggi/nausicaa/blob/master/sqlite/src/libraries/foreign/
databases/sqlite/sizeof.sls.in>

What is the expected output? What do you see instead?
It should load the library with no error. It segfaults printing:

GC Warning: Out of Memory!  Returning NIL!
Segmentation Fault

What version of the product are you using? On what operating system?
Mosh revision 2171. i686-pc-linux-gnu.

Please provide any additional information below.
As you can see, it is a GNU Autoconf input file where all the "@...@" are
substituted with actual values. This is done correctly, and the library can
be loaded and executed correctly by Ikarus and Ypsilon. The problem (I
established through trial and error) is the big set of macros:

(define-c-struct-accessor
  sqlite3-api-routines-aggregate_context-ref
  @OFFSETOF_SQLITE_API_ROUTINES_AGGREGATE_CONTEXT@
  @GETTEROF_SQLITE_API_ROUTINES_AGGREGATE_CONTEXT@)

which becomes as follows after Autoconf processing:

(define-c-struct-accessor
  sqlite3-api-routines-aggregate_context-ref
  0
  pointer-ref-c-pointer)

If I remove all these definitions and the associated identifiers in the
export list, everything goes fine and the library is loaded.  The macro
definition is:

(define-syntax define-c-struct-accessor
  (lambda (use-stx)
    (syntax-case use-stx ()
      ((_ ?accessor-name ?field-offset ?foreign-type-accessor)
       (if (syntax->datum (syntax ?field-offset))
       #'(define-syntax ?accessor-name
           (syntax-rules ()
         ((_ struct-pointer)
          (?foreign-type-accessor struct-pointer ?field-offset))))
     #'(define-syntax ?accessor-name
         (syntax-rules ()
           ((_ struct-pointer)
        (raise-unimplemented-error (quote ?accessor-name))))))))))

so it expands to a macro definition like

(define-syntax sqlite3-api-routines-aggregate_context-ref
  (syntax-rules ()
     ((_ struct-pointer)
      (pointer-c-ref-pointer struct-pointer 0))))

For some reason this ends up in a runaway memory allocation.  This is the
only library showing this behaviour, all my other libraries seem to
evaluate fine (and some of them have a lot of definitions, but none so many
macro definitions).

Random attempts:

* Commenting out the "sqlite3-api-routines-*" identifiers from the export
list and leaving in the macro definitions changes nothing, it still segfaults.

* There are 154 identifiers like "sqlite3-api-routines-*"; commenting out
some of them, I get random, unreproducible, errors like:

 Condition components:
 1. &assertion
 2. &who             who: "car"
 3. &message         message: "pair required"
 4. &irritants       irritants: (bytevector-ieee-single-ref)

 Exception:
     error in raise: returned from non-continuable exception

 Stack trace:
    1. throw: <subr>

and other ones like:

 Condition components:
 1. &assertion
 2. &who             who: "simple-struct-ref"
 3. &message         message: "simple-struct required"
 4. &irritants       irritants: (#(1 Yd8a@x #f 3 0))

 Exception:
     error in raise: returned from non-continuable exception

 Stack trace:
    1. throw: <subr>

* I dunno if the problem is the big number of macros or the big number of
identifiers in the export list.  Commenting out "many" macros and "many"
exported identifiers, the segfault goes away and the library loads fine. 
It is not a problem with the specific macro expansion.

Wild guess: Try to force a garbage collection, every now and then.  Anyway,
I am available for further testing; in particular, I can test it with a
previous revision, if you indicate me which one can be useful.

Enjoy! ;-)

Original issue reported on code.google.com by mrc....@gmail.com on 2 Nov 2009 at 10:11

[GoogleCodeExporter]

Hi. Thank you for your report.
I think I reproduced the bug.

** install
sudo apt-get install libsqlite3-dev texinfo

cd arcomaggi-nausicaa-dbe5731/scheme
autoconf
mkdir build
cd build
../configure --enable-mosh --prefix=~/tmp/nausicaa && make && make install

cd arcomaggi-nausicaa-dbe5731/sqlite/
autoconf
mkdir build
cd build
../configure --enable-mosh --prefix=~/tmp/nausicaa && make && make install

** work.scm
% cat work.scm
(import (rnrs)
        (foreign databases sqlite sizeof))  

** run
% ./mosh --loadpath=~/tmp/nausicaa/lib/nausicaa/ ./work.scm
*** ASSERT failure VM-Run.cpp:956: pcObject.isObjectPointer()
    dc_ = #<closure>
    cl_=(match e p m* s* ae* r) :psyntax.scm:10

Yes, something bad happens.

Original comment by hige...@gmail.com on 2 Nov 2009 at 2:45

• Changed state: Accepted

[GoogleCodeExporter]

Original comment by hige...@gmail.com on 17 Nov 2009 at 3:10

[GoogleCodeExporter]

Hi.
Today I tried to reproduce the bug again, but can't.

I've checked on Mosn trunk revision 2171(which you told me), 2181, 2185, and 
HEAD(2187).

Recently I upgraded my Ubuntu, it may also upgrade g++ to 4.4.0.
Or did you change something on nausicaa around sqlite?

Original comment by hige...@gmail.com on 18 Nov 2009 at 3:26

[GoogleCodeExporter]

Original comment by hige...@gmail.com on 18 Nov 2009 at 3:34

• Added labels: Milestone-Release0.3.0

[GoogleCodeExporter]

> Or did you change something on nausicaa around sqlite?

Yes, I commented out everything related to the inspection of
the structure "sqlite3_api_routines" (I was plain wrong in
thinking that the structure is needed to implement custom
SQL functions, *shame*). The attached tarball should reproduce
this bug with mosh.

Original comment by mrc....@gmail.com on 18 Nov 2009 at 6:08

Attachments:

• nausicaa-sqlite-0.1b0-mosh-bug.tar.bz2

[GoogleCodeExporter]

Thank you.

>I was plain wrong in
>thinking that the structure is needed to implement custom
>SQL functions, *shame*

This means I can make this bug pending for a while :D.

Original comment by hige...@gmail.com on 18 Nov 2009 at 6:16

[GoogleCodeExporter]

This bug raises its ugly head with my binding to Libcurl, too. I did some 
research
with the code in the attached tarball; notice that the shell scripts always 
wipe out
the Mosh' cache in "~/.mosh" before running mosh.  It seems that no matter how 
many
definitions like:

(define D1 1)

I put in a library, Mosh never crashes; but when I fill a library with 
definitions like:

(define-syntax D1
  (syntax-rules ()
    ((_ ?a ?b)
     (list 1 ?a ?b))))

I can reliably cause a crash; according to my experiments, Mosh crashes when 
there
are 746 or more such definitions and it does not crash when there are 745 or 
less
such definitions.

Original comment by mrc....@gmail.com on 20 Nov 2009 at 7:10

Attachments:

• mosh-issue-112.tar.gz

[GoogleCodeExporter]

I get the following stack trace:

GC Warning: Out of Memory!  Returning NIL!
[New Thread 0xb7d976d0 (LWP 12411)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb7d976d0 (LWP 12411)]
0xb7e0b31f in memcpy () from /lib/libc.so.6
(gdb) bt 10
#0  0xb7e0b31f in memcpy () from /lib/libc.so.6
#1  0x08063b1f in scheme::Object::makeStack (src=0xb263000, size=-5252074) at 
Stack.h:45
#2  0x08056db7 in scheme::VM::run (this=0x9e58e00, code=0xa1bb000, 
returnPoint=0x0,
returnTable=false)
    at VM-Run.cpp:452
#3  0x08051617 in scheme::VM::activateR6RSMode (this=0x9e58e00, 
isDebugExpand=false)
at VM.cpp:368
#4  0x0804d0e1 in main (argc=165962368, argv=0x9fd89f0) at main.cpp:268

the negative "size" argument to "makeStack()" does not look right to me.

Original comment by mrc....@gmail.com on 20 Nov 2009 at 8:03

[GoogleCodeExporter]

Wow. Thanks.

Original comment by hige...@gmail.com on 22 Nov 2009 at 5:10

[GoogleCodeExporter]

I think this bug is fixed on revision 2190 trunk.
Your code and stack trace are very good hints.

Thank you.

Original comment by hige...@gmail.com on 23 Nov 2009 at 8:47

• Changed state: Fixed

[GoogleCodeExporter]

I confirm that my curl library runs, along with the raw tests. (mosh -V in 
revision
2190 still believes to be revision 2186.)

Original comment by mrc....@gmail.com on 23 Nov 2009 at 9:28

[GoogleCodeExporter]

Thank you.
Sorry for version.

Original comment by hige...@gmail.com on 23 Nov 2009 at 9:49