Bad memory access when using xmlsec method.

I've been toying around with this extension and cannot get the signing or encryption methods to work. :crying_cat_face:

The extension will build just fine but a call into libxml2 (inside of libxmlsec1) end up segfaulting and it's not entirely clear why this is happening.

Here's a crash report

/Users/sbryant/github/xmlsec/lib/xmlsec.rb:23: [BUG] Segmentation fault at 0x007fac85c75e14
ruby 2.1.1p76-github (development) [x86_64-darwin13.0]

-- Crash Report log information --------------------------------------------
   See Crash Report log file under the one of following:
     * ~/Library/Logs/CrashReporter
     * /Library/Logs/CrashReporter
     * ~/Library/Logs/DiagnosticReports
     * /Library/Logs/DiagnosticReports
   for more details.

-- Control frame information -----------------------------------------------
c:0004 p:---- s:0017 e:000016 CFUNC  :sign_with_key
c:0003 p:0110 s:0012 e:000011 METHOD /Users/sbryant/github/xmlsec/lib/xmlsec.rb:23
c:0002 p:0093 s:0007 E:001040 EVAL   test.rb:9 [FINISH]
c:0001 p:0000 s:0002 E:001748 TOP    [FINISH]

test.rb:9:in `<main>'
/Users/sbryant/github/xmlsec/lib/xmlsec.rb:23:in `sign!'
/Users/sbryant/github/xmlsec/lib/xmlsec.rb:23:in `sign_with_key'

-- C level backtrace information -------------------------------------------
0   ruby                                0x0000000108c59996 rb_vm_bugreport + 134
1   ruby                                0x0000000108b0fd73 report_bug + 307
2   ruby                                0x0000000108b0fc34 rb_bug + 180
3   ruby                                0x0000000108bdd0d9 sigsegv + 153
4   libsystem_platform.dylib            0x00007fff868f95aa _sigtramp + 26
5   libxml2.2.dylib                     0x00000001094fc2df xmlDictLookup + 179
6   ???                                 0x202c6f6c6c654820 0x0 + 2318350419654690848

I've also setup the library with some debug symbols. Here are my changes to the extconf.rb:

diff --git a/ext/nokogiri_ext_xmlsec/extconf.rb b/ext/nokogiri_ext_xmlsec/extconf.rb
index 1859212..bb86802 100644
--- a/ext/nokogiri_ext_xmlsec/extconf.rb
+++ b/ext/nokogiri_ext_xmlsec/extconf.rb
@@ -6,13 +6,20 @@ end

 barf unless have_header('ruby.h')

+$CFLAGS = CONFIG['CFLAGS'].gsub(/\s\-O\d?\s/, ' -O0 ')
+$CFLAGS = "#{$CFLAGS} -g"
+$CFLAGS.gsub!(/\s?\-g\w*\s/, ' -ggdb3 ')
+CONFIG['LDSHARED'] = CONFIG['LDSHARED'].gsub(/\s\-s(\s|\z)/, ' ')
+
+$LDFLAGS = "-L/opt/boxen/homebrew/lib -L/opt/boxen/homebrew/opt/libxml2/lib #{$LDFLAGS}"
+$CFLAGS = "-L/opt/boxen/homebrew/lib -I/opt/boxen/homebrew/include -I/opt/boxen/homebrew/include/xmlsec1 -I/opt/boxen/homebrew/opt/libxml2/include/libxml2 #{$CFLAGS}"
+
 if pkg_config('xmlsec1-openssl')
   # HACK 'openssl' is escaped too many times, I don't know why
   if $CFLAGS =~ /\-DXMLSEC_CRYPTO=\\\\\\"openssl\\\\\\"/
     $CFLAGS['-DXMLSEC_CRYPTO=\\\\\\"openssl\\\\\\"'] =
       '-DXMLSEC_CRYPTO=\\"openssl\\"'
   end
-  
   have_library 'xmlsec1-openssl'
   create_makefile('nokogiri_ext_xmlsec')
 else

And found the call to xmlSecTmplSignatureCreate ends up with a call to xmlDictLookup and it does not like the data being passed in.

This made no sense to me as the code the signing code pretty much the signing example included in the xmlsec1 library. My only guess was the xml document struct being retrieved from the class isn't valid at some point.

I wondered if the same document but not managed by ruby, when running through the same code, would encounter the same issues. The answer was no, I used the following diff to test the idea out:

diff --git a/ext/nokogiri_ext_xmlsec/nokogiri_sign_rsa.c b/ext/nokogiri_ext_xmlsec/nokogiri_sign_rsa.c
index a06af38..9e24338 100644
--- a/ext/nokogiri_ext_xmlsec/nokogiri_sign_rsa.c
+++ b/ext/nokogiri_ext_xmlsec/nokogiri_sign_rsa.c
@@ -1,4 +1,5 @@
 #include "xmlsecrb.h"
+#include <stdio.h>

 // TODO the signature context probably should be a ruby instance variable
 // and separate object, instead of being allocated/freed in each method.
@@ -13,14 +14,24 @@ VALUE sign_with_key(VALUE self, VALUE rb_key_name, VALUE rb_rsa_key) {
   char *rsaKey;
   unsigned int rsaKeyLength;

-  Data_Get_Struct(self, xmlDoc, doc);
   rsaKey = RSTRING_PTR(rb_rsa_key);
   rsaKeyLength = RSTRING_LEN(rb_rsa_key);
   keyName = RSTRING_PTR(rb_key_name);

+  xmlDocPtr dup;
+  Data_Get_Struct(self, xmlDoc, dup);
+
+  doc = xmlCopyDoc(dup, 1);
+
+  if ((doc == NULL) || (xmlDocGetRootElement(doc) == NULL)){
+      rb_raise(rb_eSigningError, "No document found.");
+      goto done;
+  }
+
   // create signature template for RSA-SHA1 enveloped signature
   signNode = xmlSecTmplSignatureCreate(doc, xmlSecTransformExclC14NId,
                                          xmlSecTransformRsaSha1Id, NULL);
+
   if (signNode == NULL) {
     rb_raise(rb_eSigningError, "failed to create signature template");
     goto done;
@@ -86,6 +97,12 @@ VALUE sign_with_key(VALUE self, VALUE rb_key_name, VALUE rb_rsa_key) {
     goto done;
   }

+  xmlChar *xmlbuff;
+  int buffersize;
+  xmlDocDumpFormatMemory(doc, &xmlbuff, &buffersize, 1);
+  printf("%s", (char *) xmlbuff);
+  fflush(stdout);
+
 done:
   if(dsigCtx != NULL) {
     xmlSecDSigCtxDestroy(dsigCtx);

The following is the driver I was using to help debug the issue through lldb (on OS X here):

require 'bundler/setup'
require 'rubygems'
require 'nokogiri-xmlsec'

sign_xml = File.read('spec/fixtures/sign2-doc.xml')
priv_key = File.read('spec/fixtures/rsa.pem')
sign_doc = Nokogiri::XML(sign_xml)
sign_doc.sign!(:key => priv_key, :name => 'test')
puts sign_doc.to_xml

Any thoughts as to what could be causing these issues?

sinisterchipmunk / xmlsec

Bad memory access when using xmlsec method. #3