Specs failing on macOS Sierra and High Sierra

[shaunakpp]

On a typical macOS machine, whenever we run make after running ruby extconf.rb, .bundle and .o files are created. .bundle is the macOS equivalent of .so files. So, requiring them in our ruby programs should expect the standard behaviour.

But, when I run the basic_ruby_method.spec,which runs basic_ruby_method.rubex, requiring the .bundle file created after running make doesn't work and gives the following error:

TypeError: wrong argument type false (expected Class)

Here is the C file generated by rubex:

/* C extension for basic_ruby_method.
This file in generated by Rubex::Compiler. Do not change!
File generation time: 2017-10-03 17:26:38 +0530.*/

#include <ruby.h>
#include <stdint.h>
#include <stdbool.h>

#define __rubex_INT2BOOL(arg) (arg ? Qtrue : Qfalse)

VALUE rb_cObject;
static VALUE __rubex_rb_f_Object_addition (int argc,VALUE* argv,VALUE __rubex_arg_self);

VALUE __rubex_char2rubystr(char ch);
VALUE __rubex_char2rubystr(char ch)
{
  char s[2];
  s[0] = ch;
  s[1] = '\0';
  return rb_str_new2(s);
}

static VALUE __rubex_rb_f_Object_addition (int argc,VALUE* argv,VALUE __rubex_arg_self)
{
  int32_t __rubex_arg_a;
  int32_t __rubex_arg_b;
  VALUE __rubex_v_t;
  VALUE __rubex_temp_1;
  if (argc < 2)
  {
    rb_raise(rb_eArgError, "Need 2 args, not %d", argc);
  }

  __rubex_arg_a=(int32_t)NUM2INT(argv[0]);
  __rubex_arg_b=(int32_t)NUM2INT(argv[1]);

  /* Rubex file location: /path/to/rubex/file:2 */
  __rubex_temp_1 = rb_str_new2("aaa");
  __rubex_v_t = __rubex_temp_1;
  __rubex_temp_1 = 0;
  __rubex_temp_1 = 0;

  /* Rubex file location: /path/to/rubex/file:3 */
  rb_funcall(__rubex_v_t, rb_intern("nil?"), 0, NULL);

  /* Rubex file location: /path/to/rubex/file:4 */
  return INT2NUM(( __rubex_arg_a + __rubex_arg_b ));
}

void Init_basic_ruby_method ();
void Init_basic_ruby_method ()
{

  rb_define_method(rb_cObject ,"addition", __rubex_rb_f_Object_addition, -1);
}

Environment: os: macOS Sierra ruby: 2.4.1 running on rvm running gcc --version gives the following output:

Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 9.0.0 (clang-900.0.37)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

As a side note:

I tried the simple example of creating Ruby extensions in 5 minutes and it works perfectly.

[tessi]

I looked into this issue. It is because of two distinct errors. I fixed the first in #33 and are on my way to debug the other.

Issue 1 (the one fixed in #33) is because we don't cleanup the .bundle files properly in spec_helper.rb.

Issue 2 is because of the generated C code:

#include <ruby.h>
#include <stdint.h>
#include <stdbool.h>

#define __rubex_INT2BOOL(arg) (arg ? Qtrue : Qfalse)

VALUE rb_cObject;
static VALUE __rubex_rb_f_Object_addition (int argc,VALUE* argv,VALUE __rubex_arg_self);

/// ...

static VALUE __rubex_rb_f_Object_addition (int argc,VALUE* argv,VALUE __rubex_arg_self)
{
  // ...
  return INT2NUM(( __rubex_arg_a + __rubex_arg_b ));
}

void Init_basic_ruby_method ();
void Init_basic_ruby_method ()
{
  rb_define_method(rb_cObject ,"addition", __rubex_rb_f_Object_addition, -1);
}

The line VALUE rb_cObject; shadows a global variable (the reference to the ruby Object class) and sets it to an uninitialized value. In line rb_define_method(rb_cObject ,"addition", __rubex_rb_f_Object_addition, -1); (to the bottom of the file) we use rb_cObject which now instead of the ruby-Object-class holds that uninitialized value (i guess it gets set to 0 by the compiler).

Since (I guess) rb_cObject is initialized with 0 it equals the false constant. This is the reason why we get the error:

TypeError: wrong argument type false (expected Class)

We expect a class (the ruby Object class), but get false (the uninitialized value).

If I manually remove the line VALUE rb_cObject; before compilation, the test is green.

So the next step to solve this puzzle is to find out why VALUE rb_cObject; is generated.

[shaunakpp]

@v0dro can you help us out here?

[v0dro]

Ah. The problem is that a redeclaration of rb_cObject is happening here in spite of the variable being declared inside ruby.h. Its not being set to zero or anything, its just that I think clang does not allow redeclaration of variables that have already been declared inside a header file.

You can see the code that emits the class variables here: https://github.com/SciRuby/rubex/blob/master/lib/rubex/ast/node.rb#L91

It should have a check that the Object class (or for that matter any stdlib class like rb_cArray) should not be redeclared in the file.

[shaunakpp]

It should have a check that the Object class (or for that matter any stdlib class like rb_cArray) should not be redeclared in the file.

So I tried to do this by adding simple check in the file at the above mentioned line:

code << "VALUE #{klass.c_name};" unless Rubex::DEFAULT_CLASS_MAPPINGS.has_key?(klass.name)

Doing this makes most of the specs pass but, https://github.com/SciRuby/rubex/blob/master/spec/struct_spec.rb#31 fails and crashes the Ruby interpreter. I can share the logs if you want, but they're quite exhaustive.

Here are the first few lines:

/**/**/**/rubex/spec/struct_spec.rb:31: [BUG] Illegal instruction at 0x0000010d046e74
ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-darwin16]

-- Crash Report log information --------------------------------------------
   See Crash Report log file under the one of following:                    
     * ~/Library/Logs/DiagnosticReports                                     
     * /Library/Logs/DiagnosticReports                                      
   for more details.                                                        
Don't forget to include the above Crash Report log file in bug reports.     

-- Control frame information -----------------------------------------------
c:0040 p:---- s:0205 e:000204 CFUNC  :struct_ptr
c:0039 p:0109 s:0201 e:000199 BLOCK  /**/**/**/rubex/spec/struct_spec.rb:31 [FINISH]
c:0038 p:0037 s:0196 e:000195 METHOD /**/**/**/rubex/spec/spec_helper.rb:51
c:0037 p:0014 s:0188 E:0023f0 BLOCK  /**/**/**/rubex/spec/struct_spec.rb:25 [FINISH]
c:0036 p:---- s:0185 e:000184 CFUNC  :instance_exec

[v0dro]

Can you also share the generated C code for that particular spec?

[shaunakpp]

/* C extension for struct.
This file in generated by Rubex::Compiler. Do not change!
File generation time: 2018-02-01 10:06:22 +0530.*/

#include <ruby.h>
#include <stdint.h>
#include <stdbool.h>

#define __rubex_INT2BOOL(arg) (arg ? Qtrue : Qfalse)

static VALUE __rubex_rb_f_Object_structure (int argc,VALUE* argv,VALUE __rubex_arg_self);
static VALUE __rubex_rb_f_Object_struct_index (int argc,VALUE* argv,VALUE __rubex_arg_self);
static VALUE __rubex_rb_f_Object_access_struct_obj (int argc,VALUE* argv,VALUE __rubex_arg_self);
static VALUE __rubex_rb_f_Object_struct_ptr (int argc,VALUE* argv,VALUE __rubex_arg_self);

VALUE __rubex_char2rubystr(char ch);
VALUE __rubex_char2rubystr(char ch)
{
  char s[2];
  s[0] = ch;
  s[1] = '\0';
  return rb_str_new2(s);
}

static VALUE __rubex_rb_f_Object_structure (int argc,VALUE* argv,VALUE __rubex_arg_self)
{
  typedef struct other_node __rubex_t_Object_other_node;
  typedef struct node
  {
    int32_t __rubex_v_a;
    int32_t __rubex_v_b;
    __rubex_t_Object_other_node* __rubex_ptr_hello;
    float __rubex_arr_c[10];
  } __rubex_t_Object_node;

  typedef struct other_node
  {
    int64_t __rubex_v_a;
    int64_t __rubex_v_b;
    char* __rubex_ptr_string;
  } __rubex_t_Object_other_node;

  typedef struct empty_struct
  {
  } __rubex_t_Object_empty_struct;

  VALUE __rubex_arg_a;
  int32_t __rubex_arg_b;
  int32_t __rubex_arg_c;
  __rubex_t_Object_node __rubex_v_n;
  __rubex_t_Object_other_node __rubex_v_shell;
  int32_t __rubex_v_i;
  int32_t __rubex_arr_f[10];
  if (argc < 3)
  {
    rb_raise(rb_eArgError, "Need 3 args, not %d", argc);
  }

  __rubex_arg_a=argv[0];
  __rubex_arg_b=(int32_t)NUM2INT(argv[1]);
  __rubex_arg_c=(int32_t)NUM2INT(argv[2]);

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:21 */
  __rubex_v_n.__rubex_v_a = __rubex_arg_b;

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:22 */
  __rubex_v_n.__rubex_v_b = __rubex_arg_c;

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:26 */
  __rubex_v_shell.__rubex_ptr_string = StringValueCStr(__rubex_arg_a);

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:27 */
  __rubex_v_shell.__rubex_v_a = 666;

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:28 */
  __rubex_v_shell.__rubex_v_b = 555;
  for (__rubex_v_i = 0; __rubex_v_i < 10; __rubex_v_i++)
  {

    /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:32 */
    __rubex_v_n.__rubex_arr_c[__rubex_v_i] = ( __rubex_v_i * 43 );

    /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:33 */
    __rubex_arr_f[__rubex_v_i] =     ( __rubex_v_i + 4 );
  }

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:36 */
  printf("%f", __rubex_v_n.__rubex_arr_c[7]);

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:38 */
  return LONG2NUM(__rubex_v_shell.__rubex_v_a);
}

static VALUE __rubex_rb_f_Object_struct_index (int argc,VALUE* argv,VALUE __rubex_arg_self)
{
  typedef struct node
  {
    int __rubex_v_a;
    int __rubex_arr_ptr[20];
    int __rubex_arr_ptr1[5];
  } __rubex_t_Object_node;

  int __rubex_v_i;
  __rubex_t_Object_node __rubex_v_n;
  if (argc < 0)
  {
    rb_raise(rb_eArgError, "Need 0 args, not %d", argc);
  }

  __rubex_v_i = 0;
  for (__rubex_v_i = 0; __rubex_v_i < 20; __rubex_v_i++)
  {

    /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:51 */
    __rubex_v_n.__rubex_arr_ptr[__rubex_v_i] = __rubex_v_i;
  }

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:53 */
  __rubex_v_n.__rubex_v_a = 4;

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:54 */
  __rubex_v_n.__rubex_arr_ptr1[4] = 4;

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:56 */
  return INT2NUM(__rubex_v_n.__rubex_arr_ptr[__rubex_v_n.__rubex_arr_ptr1[__rubex_v_n.__rubex_v_a]]);
}

static VALUE __rubex_rb_f_Object_access_struct_obj (int argc,VALUE* argv,VALUE __rubex_arg_self)
{
  typedef struct node
  {
    VALUE __rubex_v_array;
  } __rubex_t_Object_node;

  __rubex_t_Object_node __rubex_v_a;
  VALUE __rubex_temp_1;
  if (argc < 0)
  {
    rb_raise(rb_eArgError, "Need 0 args, not %d", argc);
  }

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:66 */
  __rubex_temp_1 = rb_ary_new2(0);
  __rubex_v_a.__rubex_v_array = __rubex_temp_1;
  __rubex_temp_1 = 0;

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:67 */
  rb_funcall(__rubex_v_a.__rubex_v_array, rb_intern("push"), 1 ,INT2NUM(1));

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:69 */
  __rubex_temp_1 = rb_funcall(__rubex_v_a.__rubex_v_array, rb_intern("[]"), 1, INT2NUM(0));
  return __rubex_temp_1;
}

static VALUE __rubex_rb_f_Object_struct_ptr (int argc,VALUE* argv,VALUE __rubex_arg_self)
{
  typedef struct node
  {
    int __rubex_v_i;
    int __rubex_v_j;
  } __rubex_t_Object_node;

  __rubex_t_Object_node* __rubex_ptr_a;
  if (argc < 0)
  {
    rb_raise(rb_eArgError, "Need 0 args, not %d", argc);
  }

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:78 */
  __rubex_ptr_a->__rubex_v_i = 3;

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:79 */
  __rubex_ptr_a->__rubex_v_j = 4;

  /* Rubex file location: /rubex/spec/fixtures/struct/struct.rubex:81 */
  return INT2NUM(( __rubex_ptr_a->__rubex_v_i + __rubex_ptr_a->__rubex_v_j ));
}

void Init_struct ();
void Init_struct ()
{

  rb_define_method(rb_cObject ,"structure", __rubex_rb_f_Object_structure, -1);
  rb_define_method(rb_cObject ,"struct_index", __rubex_rb_f_Object_struct_index, -1);
  rb_define_method(rb_cObject ,"access_struct_obj", __rubex_rb_f_Object_access_struct_obj, -1);
  rb_define_method(rb_cObject ,"struct_ptr", __rubex_rb_f_Object_struct_ptr, -1);
}

[shaunakpp]

Oh, and one more thing, I've further narrowed down the error to the function __rubex_rb_f_Object_struct_ptr. Rest of the methods seem to work fine!

[v0dro]

Ah. Can you try adding a malloc(sizeof(node)) call at the allocation statement in the struct_ptr function? I mean in the Rubex code.

[shaunakpp]

Yup, had to do malloc to make it work.

  __rubex_t_Object_node* __rubex_ptr_a = malloc(sizeof(__rubex_t_Object_node));

[v0dro]

Latest master solves this issue.