xar command line signature support (patch included)

[GoogleCodeExporter]

Support signing xar archives from the command line using the xar command line 
tool.

The attached patch applies cleanly to truck@232 and is heavily based on code 
from:

http://opensource.apple.com/source/xar/xar-36.1/xarsig/xar-sig.c

-----
An xar tool with signature support can be built and installed like so:

svn export http://xar.googlecode.com/svn/trunk/xar@232 xar_232
patch -d xar_232 -p0 < xar_debug_makefile.patch
patch -d xar_232 -p0 < xar_cli_signature_232.patch
cd xar_232
./autogen.sh --noconfigure
PATH="`getconf PATH`:/usr/local/bin" CFLAGS='-Wall -Os' ./configure 
--without-lzma

If make is GNU make (try make --version to find out) use:

make
sudo make install

If gmake is GNU make (i.e. FreeBSD, try gmake --version to find out) use:

gmake
sudo gmake install

-----
Patch has been tested and is known to work correctly on Mac OS X, Ubuntu and 
FreeBSD.

-----
After building an xar tool with signature support, an xar archive (including 
Apple's .pkg flat-file format) can be signed like so (example signing commands 
written for sh-compatible shell):

1. Assume the leaf certificate in DER format is stored in leaf.der and the CA 
intermediate certificate that issued that leaf certificate is stored in DER 
format in intCA.der.  Also assume the archive to sign is archive.xar and the 
RSA private key for leaf.der is stored in PEM format in key.pem.

2. Determine the signature size (depends on the RSA private key bit size):

  : | openssl dgst -sign key.pem -binary | wc -c > siglen.txt

3. Extract the hash to sign and insert the certificate data:

  xar --sign -f archive.xar --data-to-sign sha1_hash.dat \
  --sig-size `cat siglen.txt` --cert-loc leaf.der --cert-loc intCA.der

NOTE: only the leaf signing certificate must be given (it should be the first 
--cert-loc option), zero or more additional --cert-local options may be used to 
embed intermediary certificates that lead up to the trusted root certificate 
that the client is expected to already have installed.

4. Generate the signature:

  (echo "3021300906052B0E03021A05000414" | xxd -r -p; cat sha1_hash.dat) |
  openssl rsautl -sign -inkey key.pem > signature.dat

5. Inject the signature into the archive:

  xar --inject-sig signature.dat -f archive.xar

6. Remove the temporary files:

  rm siglen.txt sha1_hash.dat signature.dat

NOTE: The leaf certificate must use RSA with SHA1 for signatures and the xar 
archive must use SHA1 checksum hashes (the default).

Original issue reported on code.google.com by mack...@gmail.com on 16 Aug 2010 at 7:15

Attachments:

• xar_cli_signature_232-patch.txt

[GoogleCodeExporter]

Hello mackyle!

I tried your instructions but run into some issues... let's see if you can help.

1)I did not have any "xar_debug_makefile.patch" so I skipped this part. hope 
that's fine. everything else went fine so I guess it was.

2) Is '3021300906052B0E03021A05000414' a random thing or is it supposed to mean 
something?

3) This is the most important point:

I can sign and extract signatures from a xar that i create (i'm on Ubunut).

But I cannot work on a xar file coming from mac os x, and more in particular, I 
cannot work over a .safariextz Safari extension package!

This is actually what I was hoping to be able to do - develop safari extensions 
on my linux machine.

But trying to read (signed) Safari extension packages brings checksum errors, 
and signed xars are unaccepted by safari.

As an example: http://defacer.googlecode.com/files/Defacer1.0.3.safariextz

   --dumb-header WORKS

xar --dump-header -f Defacer.safariextz
magic:                  0x78617221 (OK)
size:                   28
version:                1
Compressed TOC length:  4912
Uncompressed TOC length: 14607
Checksum algorithm:     1 (SHA1)

    -t, -x, --extract-sig ... DO NOT work

xar --extract-sig testsig.pem -f Defacer.safariextz
Checksums do not match!
Could not open Defacer.safariextz to extract signature data!

Any idea?!

thanks!

Original comment by stefano....@gmail.com on 25 Aug 2010 at 2:58

[GoogleCodeExporter]

A1. Sorry, meant to remove the xar_debug_makefile.patch patch item from the 
instructions.  It's not needed unless you want to run the debugger.  You can 
get it from Bug 75 at:

http://code.google.com/p/xar/issues/detail?id=75

where it's named "Makefile.inc.in-patch.txt" instead of 
"xar_debug_makefile.patch".  I've also attached it here as 
"xar_debug_makefile-patch.txt".  But again, it's not needed for the signing 
patch to work.

A2. '3021300906052B0E03021A05000414' has a very specific meaning.  See:

http://tools.ietf.org/html/rfc3447#section-9.2

In section 9.2 scroll on down to the "Notes." and in note "1." see the value 
for SHA-1.

A3. The file that's located at the listed defacer URL above now has a correct 
checksum (and the output of --dump-header is different than what you have 
listed.)

When you see "Checksums do not match!" it means the checksum of the raw 
table-of-contents data does not match the checksum stored in the xar file at 
the offset listed in the table of contents.

You can check this by dumping the header, adding together the "size:" and the 
"Compressed TOC length:" values and then dumping 20 bytes at that offset in the 
xar file (assuming the checksum offset in the header is 0 which it almost 
always is otherwise you have to add that value too).

Compare that sha1 hash value to the value generated by doing:

xar --dump-toc-raw=rawtoc.dat -f some_xar_file.xar
openssl sha1 < rawtoc.dat

(FYI, you can get the checksum out of the .xar file using something like:

dd if=some_xar_file.xar bs=1 skip=sum_of_offset_and_compressed_toc_length 
count=20 of=checksum.dat

and then run xxd checksum.dat to see it so you can compare it to the value 
generated by the openssl command.
)

One other thing, when OS X is dealing with xar files if something's wrong with 
the checksum or the signed checksum differs all you seem to get for an error is 
something like "could not open" rather than useful information such as bad 
checksum or bad signature etc.

Original comment by mack...@gmail.com on 26 Aug 2010 at 6:29

Attachments:

• xar_debug_makefile-patch.txt

[GoogleCodeExporter]

mackyle, 
thank you so much for taking the time to answer; let's see if you feel like 
wasting a bit more time with me!

Perfect for A1 and A2.

Concerning my checksum error, you did clarify the magic behind it perfectly, 
but I still get the error on any signed '.safariextz' package (including a 
fresh download of Defacer). 
I attach another example coming from 
http://www.macosxtips.co.uk/keysearch/download 
Any command (-t, -x) will get me the error; the standard ubuntu xar 1.5.2 would 
open the package no problem. Just to be sure, i'm using three different 
machines to have a clean xar 1.5.2 and my own compiled xar 1.6dev with and 
without patch. 

Since you seem to be able to work on those archives with no problem I suspect I 
might be having some issue with my build... It has actually nothing to do with 
your patch. making again from the 232 source produces the same issue.

Any suggestion on what I might have gotten wrong? Missing some libraries, 
configure options,...?
I'm willing to try ;-) .
Or should I just open a new Issue?

PS.
I do not have a --dump-toc-raw option, only --dump-toc ... Any other way to 
extract it?

That's what I get for the checksum (both with xar 1.5.2 and xar 1.6dev)

> xar --dump-header -f keysearch.safariextz

magic:                  0x78617221 (OK)
size:                   28
version:                1
Compressed TOC length:  5700
Uncompressed TOC length: 15959
Checksum algorithm:     1 (SHA1)

> dd if=keysearch.safariextz bs=1 skip=5728 count=20 of=keysearch.checksum.dat

> xxd keysearch.checksum.dat

0000000: 5f73 c611 ce09 b518 c92e dd4c a084 f4b8  _s.........L....
0000010: 3ac5 4a14                                :.J.

again, thanks a lot for your help

Original comment by stefano....@gmail.com on 27 Aug 2010 at 9:35

Attachments:

• keysearch.safariextz
• keysearch.checksum.dat

[GoogleCodeExporter]

It's possible you have a conflict between the new xar and the xar package you 
have installed.  I have successfully built and tested the patched xar on both 
Ubuntu 10.4 i386 32-bit and Ubuntu 10.4 x86_64 64-bit and it works fine.

The xar build creates a library (libxar.so.1 on linux) and an executable (xar). 
 If you were to build the new executable and run it against the old library you 
would have problems similar to what you've described.

Check the libraries your new xar binary will load with:

  ldd xar

You must be having problems building/patching/installing the signature-capable 
xar because "--dump-toc-raw" is a new option provided as part of the signature 
patch attached to this issue.

You can check to see you are running the correct xar by looking at the output 
of "xar --help".  If you see "--dump-toc-raw=<filename>" in the help you have 
the correct binary.  If you then run "ldd xar" on that binary and see that it's 
really loading the newly installed library (should be 
/usr/local/lib/libxar.so.1 unless you used a --prefix option on configure) then 
you should be good to go.

The only configure option you need is the one shown above "--without-lzma" 
because the xar sources don't seem to compile against current lzma libraries.

In case you are having trouble applying the patches, I've attached an archive 
of the patched sources that have also already had "./autogen.sh --noconfigure" 
run on them.  You should be able to build and install by doing:

tar xvzf xar_sig_232_src.tar.gz
cd name-of-newly-extracted-directory
./configure --without-lzma
make
sudo make install

Original comment by mack...@gmail.com on 31 Aug 2010 at 1:54

Attachments:

• xar_sig_232_src.tar.gz

[GoogleCodeExporter]

Hi! I'm trying to do the same on a Mac OS X.

xar --dump-toc-raw=rawtoc.dat -f some_xar_file.xar
openssl sha1 < rawtoc.dat

and

dd if=some_xar_file.xar bs=1 skip=sum_of_offset_and_compressed_toc_length 
count=20 of=checksum.dat

are the same, but "Safari can't accept this extension". What could I miss?

Original comment by kukushec...@gmail.com on 31 Aug 2010 at 1:22

[GoogleCodeExporter]

mackyle, thanks once again for trying and helping.

Indeed I do have --dump-toc-raw, my bad; in trying and understanding my 
checksum error I had recompiled xar /without/ the patch, and expected to find 
dump-toc-raw nevertheless.

So: the checksum is fine.

For: 
xar --dump-toc-raw=keysearch.rawtoc.dat -f keysearch.safariextz
openssl sha1 < keysearch.rawtoc.dat

I do get the same 5f73c611ce09b518c92edd4ca084f4b83ac54a14 I got before, but 

xar -tf keysearch.safariextz

still gives the same 

Checksums do not match!
Error opening xar archive: keysearch.safariextz

error as before.

Hoping I'm not bothering you too much, any idea what this could be due to?

I tryed uninstalling any xar and building from your code, but I get exactly the 
same error!

kukushechkin, does this simple thing work for you? Can you open with xar a 
.safariextz package?

Original comment by stefano....@gmail.com on 31 Aug 2010 at 2:10

[GoogleCodeExporter]

 stefano.crosta, I can xar -tf myExtension.safariextz (or even -xf) without problem, but Safari cannot install it.

Original comment by kukushec...@gmail.com on 31 Aug 2010 at 2:38

[GoogleCodeExporter]

Also! I compared .safariextz, generated by Safari and manually, both have equal 
checksum, but it differs between them, which seems odd to me as they were 
signed with the same certificate.

Original comment by kukushec...@gmail.com on 31 Aug 2010 at 2:43

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

(deleted prev comment to fixe some typos) 

argh I wish I knew what goes wrong with my xar.

I now just built a new xar on yet another ubuntu 10.4 and this time I don't 
have the checksum error no more. Good, though I'd still like to understand what 
was wrong on the other machine. I had no build errors.

So I generate again a signed safariextz and got the same error I started with: 
"Safari can't install this extension. An error occurred while installing this 
extension.". 

I figure the generated xar package is somehow not compliant with apple one, and 
know I do not have to worry about checksum errors on my side any more, so I can 
explore some more.

In case it might help, I attach my test (and btw I did not develop 
keywordsearch, it's just a test!) :

 * the 'keysearch.signed' extension that I extract, re-package with xar and sign with my certificate

 * the 'keysearch.xbsigned' extension re-signed with my certificate using safari extension builder

Differences I can spot in the TOC:
  * I did not include in my signing the root certificate (might that be an issue?)
  * there's a lot of meta generated by my xar that there is not in the apple one (all file attributes: time, owner, etc.)`

and with that different of course also byte 20 to 256 are different (am I right 
assuming that's where the signature is stored?)

does this help?

mackyle have you ever been able to specifically signing a safari extension and 
have it accepted?

Original comment by stefano....@gmail.com on 31 Aug 2010 at 3:57

Attachments:

• keywordsearch.signed.safariextz
• keywordsearch.xbsigned.safariextz

[GoogleCodeExporter]

stefano.crosta,

did you try to add root certificate? And how do you obtain .pem and .der?

Original comment by kukushec...@gmail.com on 31 Aug 2010 at 4:31

[GoogleCodeExporter]

stefano.crosta,

I have found and corrected some bugs in the xar_from_base64 function.  A new 
patch is attached as xar_cli_signature_232v2-patch.txt and, for convenience, a 
fully patched and autogen'd set of sources ready to be ./configure'd as 
xar_sig_232v2_src.tar.gz.

The updated patch has the following fixes:

* xar_from_base64 no longer adds extra bytes to the decoded output 
(longstanding bug)
* the version number output by xar --version is now 1.6dev_s1 to distinguish 
from earlier patches
* the extra blank line that could sometimes be inserted in the --extract-CAfile 
output certificates has been removed

I have also examined the files attached to Comment 10.  The 
keywordsearch.signed.safariextz file contains an invalid certificate.  It had 
one byte of garbage appended to it -- precisely the bug that the 
xar_from_base64 function had before the attached, updated patch.

Did you check to see that you had a valid certificate by running "openssl x509 
-noout -text -inform der -in mycertificate_file" before you passed it to xar 
with the --cert-loc option?

The xar_from_base64 bugs did not affect signing, but they did affect the output 
of --dump-toc, --extract-certs and --extract-CAfile.  Did you perhaps extract 
the certificate used to sign keywordsearch.signed.safariextz by running one of 
the --dump-toc, --extract-certs or --extract-CAfile options on 
keywordsearch.xbsigned.safariextz (or another signed xar) and then use that to 
sign keywordsearch.signed.safariextz?  That would explain how you ended up with 
a certificate with one byte of garbage data appended to the end.

FYI, you can easily extract your certificate from the Mac's Keychain Access by 
selecting the certificate and exporting it as ".cer" format (which is really 
the binary DER format that you need to pass to the --cert-loc option).

Original comment by mack...@gmail.com on 7 Sep 2010 at 2:25

Attachments:

• xar_cli_signature_232v2-patch.txt
• xar_sig_232v2_src.tar.gz

[GoogleCodeExporter]

mackyle, that's great, i'll check it out.

> did you try to add root certificate? And how do you obtain .pem and .der?
(same question from mackyle and kukushechkin)

I haven't tried adding the root certificate yet; I extracted/converted my 
certificate and key using OpenSSL from a windows pfx (I could have gotten a DER 
directly but not the pem key)

> Did you check to see that you had a valid certificate 

The certificate looks fine to me (see the attached file), and I did not use any 
xar function to extract the certificate; but I will definitely run your patch 
and try everything once again. It might not be today, but I'll do it ASAP.

thanks a lot!

Original comment by stefano....@gmail.com on 7 Sep 2010 at 9:16

Attachments:

• certificate.output.txt

[GoogleCodeExporter]

Hi, mackyle!

I've checked out patched version, but still the same. Maybe I'm doing smth 
wrong. Here's step-by-step:

1. Drag'n'drop .p12 from Keychain
2. openssl pkcs12 -in key.p12 -out key.pem -nodes
3. openssl x509 -outform der -in key.pem -out key.der (this key.der is equal to 
.cer, dragged from Keychain)
4. : | openssl dgst -sign key.pem -binary | wc -c > siglen.txt
5. xar --sign -f MyExt.safariextz --data-to-sign sha1_hash.dat --sig-size `cat 
siglen.txt` --cert-loc key.der
6. (echo "3021300906052B0E03021A05000414" | xxd -r -p; cat sha1_hash.dat) 
|openssl rsautl -sign -inkey key.pem > signature.dat
7. xar --inject-sig signature.dat -f MyExt.safariextz

Resultant .safariextz cannot be install by Safari.

Original comment by kukushec...@gmail.com on 8 Sep 2010 at 8:22

[GoogleCodeExporter]

I am looking for a way to create a .safariextz from a build system on linux.  I 
also tried this patch with same result as kukushechkin.  I did comparisons to 
what Safari produces and verified I have the same certificates.  Any help is 
appreciated. 

Original comment by drujen...@gmail.com on 9 Nov 2010 at 2:56

[GoogleCodeExporter]

I made a patch for "Checksums do not match! " and Safari5.

Original comment by crc...@gmail.com on 6 Jan 2011 at 9:21

Attachments:

• xar_cli_signature_232v2-patch-linux-safari5fix.txt

[GoogleCodeExporter]

kukushechkin & crckyl,

I've created a new patch file xar_cli_signature_232v3-patch.txt and, for 
convenience, a fully patched and autogen'd set of sources ready to be 
./configure'd as xar_sig_232v3_src.tar.gz.

The updated patch has the following fixes from crckyl's patch:

* lib/util.c now does "#define _FILE_OFFSET_BITS 64" which should help with 
checksum mismatch issues
* lib/signature.c now generates a signature-creation-time element to hopefully 
make Safari happy

And also contains the following additional changes:

* cleaned up some of the help in the xar --help output and xar man page
* xar --version now outputs "1.6dev_sigv3" to distinguish this patch from the 
earlier ones

If you really want to try and duplicate a .safariextz file you should start 
with a .safariextz file that has been signed by Safari's Extension Builder and 
extract the embedded certificate chain from it for use in the xar --sign 
command.

For example, suppose you had a Safari Extension Builder signed extension named 
"keywordsearch.xbsigned.safariextz" then you would do the following:

1. mkdir certs
2. xar --extract-certs=certs -f keywordsearch.xbsigned.safariextz

The certs directory will now contain each certificate in the chain that was 
embedded into the .safariextz xar file.  The first file "cert00" should be 
identical to your Safari signing certificate you got from Apple (Safari appears 
to be hard-coded to only accept extensions signed by Apple-issued certificates 
so don't bother trying to roll your own for development purposes).

If you perform the above two steps on the same-named file that was attached to 
comment 10 above, you will end up with a cert00, cert01 and cert02 file in the 
certs directory.  cert00 will be the leaf signing certificate (provided to you 
by Apple), cert01 will be an Apple intermediate certificate and cert02 will be 
an Apple root certificate.

All these certificates other than cert00 should be specified on the "xar 
--sign" command line in numerical order (using a --cert-loc option for each) 
AFTER the --cert-loc option that specifies the Apple-issued certificate.

For example, in comment 14 above, step #5 would be changed to:

5. xar --sign -f MyExt.safariextz --data-to-sign sha1_hash.dat --sig-size `cat 
siglen.txt` --cert-loc key.der --cert-loc certs/cert01 --cert-loc certs/cert02

Note how cert00 IS NOT specified since it's presumably a duplicate of "key.der" 
(and therefore you could actually specify certs/cert00 INSTEAD OF key.der).

Also note that the number of certificates produced by the --extract-certs 
command may vary -- it depends on how many are actually embedded in the xar 
file archive -- there will always be at least one if the archive is signed 
(extracted as cert00).  The "xar --sign" command will need to be adjusted by 
adding or removing "--cert-loc" options accordingly depending on how many 
certificates are produced by the "--extract-certs" option.

Original comment by mack...@gmail.com on 6 Jan 2011 at 6:45

Attachments:

• xar_cli_signature_232v3-patch.txt
• xar_232v3_src.tar.gz

[GoogleCodeExporter]

mackyle,

This gets me further than the previous patches, to the point where if I open 
the signed .safariextz file Safari opens and asks if I want to install it.  
However, if I try to install it, Safari crashes.  I can still install the 
Extension Builder-built extension just fine, however.

I can't seem to find any meaningful output as to why it's failing, other than:

  ...

  Application Specific Information:
  abort() called

  Thread 0 Crashed:  Dispatch queue: com.apple.main-thread
  0   libSystem.B.dylib                 0x00007fff89019616 __kill + 10
  1   libSystem.B.dylib                 0x00007fff890b9cca abort + 83
  2   libxar.1.dylib                    0x00007fff81a46399 xar_gzip_fromheap_in + 280
  3   libxar.1.dylib                    0x00007fff81a4253b xar_attrcopy_from_heap + 475
  4   libxar.1.dylib                    0x00007fff81a3ea9d xar_darwinattr_extract + 517
  5   libxar.1.dylib                    0x00007fff81a3de80 xar_arcmod_extract + 66
  6   com.apple.Safari                  0x000000010017fc78 0x100000000 + 1571960
  7   com.apple.Safari                  0x0000000100180c4b 0x100000000 + 1576011
  8   com.apple.Safari                  0x00000001001834ed 0x100000000 + 1586413
  9   com.apple.Safari                  0x0000000100183990 0x100000000 + 1587600
  10  com.apple.Safari                  0x0000000100039d55 0x100000000 + 236885
  11  com.apple.Safari                  0x000000010003987f 0x100000000 + 235647

  ...

Justin

Original comment by justin%m...@gtempaccount.com on 6 Jan 2011 at 7:23

[GoogleCodeExporter]

mackyle,

Here's some more output from Safari when it crashes from the console log:

  Safari(22604,0x7fff711a1ca0) malloc: *** mmap(size=158329674399744) failed (error code=12)
  *** error: can't allocate region
  *** set a breakpoint in malloc_error_break to debug

Justin

Original comment by justin%m...@gtempaccount.com on 6 Jan 2011 at 7:37

[GoogleCodeExporter]

Justin,

You might try adding the "--distribution" option to the xar command line when 
you build the initial xar archive to omit extraneous attributes that you don't 
really need.  This should hopefully prevent any significant use of the 
xar_darwinattr_extract function when Safari attempts to install the extension.

In other words, if your unpacked extension is in the MyExt.safariextension 
directory, use

xar -cvzf MyExt.safariextz --distribution MyExt.safariextension

Also after you've built your signed .safariextz, try extracting it using xar 
(i.e. xar -xvf MyExt.safariextz) to make sure the archive isn't corrupt.

I can't really tell what caused the above crash, but you might try looking at 
this output:

xar --dump-toc=- -f MyExt.safariextz | grep "<size>"

and see if you see any negative size values (attempting to extract the archive 
should also expose any negative size values as well) as it looks like from the 
code that if a size value was somehow negative you might see a crash.  
Hopefully though the "--distribution" option will solve the problem for you.

Original comment by mack...@gmail.com on 7 Jan 2011 at 12:14

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Hi

After compiling xar_232v3_src.tar.gz with the patchs 
xar_cli_signature_232v3-patch.txt and xar_debug_makefile-patch.txt, I got:

:$ xar --version
xar 1.6dev_sigv3

And with a super simplified process:

1. mkdir certs
2. xar --extract-certs=certs -f someExtension.safariextz
(here I compared the three cert0x with the certs in the keychain and they are 
exactly the same)

3. Then with the certificates I can in theory do something like this:
xar -c dir_with_my_extension_stuff/ -f newExtension.safariextz --sign 
--sig-size 256 --cert-loc cert00 --cert-loc cert01 --cert-loc cert02

but I get the very informative message "Safari can't install this extension\n 
An error ocurred while installing this extension"

If I do something like "xar --dum-toc=somefile -f " with both the original and 
the new xar file, I see some little differences, the main one in the tag 
offset, in the new file the offset is bigger by 15 bytes. The other differences 
are that in the new xar file some extra information related to files like 
inode, deviceno, user, mode and blah blah, are listed but not for directories. 
In the original xar, is the opposite, this kind of info is listed for 
directories but not for files. I tried the option --distribution and it fixed 
the offset, and removed almost all the extra info, and only left behind the 
"mode" tag. With --distribution the only differences are extra "mode" tag and 
the lack of information tags about directories.

And the error still remains.

Some Ideas? By the way the certificates from the keychain are ".cer". And I 
obtained the size from the original file header itself (--dump-toc), There you 
can see the offset (20), the size (256), siganture style (RSA), they all match 
with the new xar file.

Any help would be really appreciated.

Original comment by herberth...@gmail.com on 17 Jun 2011 at 5:37

[GoogleCodeExporter]

And "openssl dgst -sign key.pem -binary" did not worked on any attempt. I tried 
different ways like converting a cer into a pem or extracting the certificates 
as a pem, but no success. So, the size from the original header became very 
useful.

Original comment by herberth...@gmail.com on 17 Jun 2011 at 6:03

[GoogleCodeExporter]

@23 you didn't really sign your xar.

If you read the man page, "--sign Creates a placeholder signature and saves the 
data to sign to disk."  Steps #4 and #5 (from the top of this bug) are not 
optional.  xar does not understand how to create a real signature (it only 
understands how to manipulate xar archives), so the placeholder still left in 
the file because you skipped steps 4 and 5 does not verify and Safari refuses 
to acknowledge the file since it has an invalid signature.

"key.pem" is your private signing key.  It should be in your keychain.  Use 
Keychain Access to export it as a .p12 file.  Say it's called 
"my_private_key.p12" then you use the following to convert it to PEM format:

openssl pkcs12 -in my_private_key.p12 -nodes | openssl rsa -out key.pem

And now you have a private key in key.pem to use for steps 4 and 5.  Don't 
forget to re-add the --data-to-sign option to step 3 or you will have nothing 
to sign.

Original comment by mack...@gmail.com on 17 Jun 2011 at 9:46

[GoogleCodeExporter]

Hi

Thanks for the clarification.

Now, I almost accomplished my goal. Let me show you what I'm doing, that is 
pretty much what has been explained here.

1. mkdir certs
2. xar --extract-certs=certs -f someExtension.safariextz
3. Export certifacates in format .p12 from Keychain
4. openssl pkcs12 -in key.p12 -out key.pem -nodes
5. openssl x509 -inform PEM -in key.pem -out key.der -outform DER
6. : | openssl dgst -sign key.pem -binary | wc -c > siglen.txt
7. xar --sign -f MyExt.safariextz --data-to-sign sha1_hash.dat --sig-size `cat 
siglen.txt` --cert-loc key.der --cert-loc certs/cert01 --cert-loc certs/cert02
8. (echo "3021300906052B0E03021A05000414" | xxd -r -p; cat sha1_hash.dat) 
|openssl rsautl -sign -inkey key.pem > signature.dat
9. xar --inject-sig signature.dat -f MyExt.safariextz

I tested it and it worked pretty good. But I found one big restriction. The 
contents of MyExt.safariextz should exactly the same of 
someExtension.safariextz. In I change what is in the xar file, Safari rejects 
my extension.

Any thougts?

Original comment by herberth...@gmail.com on 22 Jun 2011 at 11:35

[GoogleCodeExporter]

You realize that you have to repeat steps 7, 8 and 9 whenever you change the 
contents of the xar file.  Just repeating step 9 would result in an invalid 
signature.  (The signature length for any one key is always the same so you 
don't need to repeat step 6 unless you change the key.)

Assuming you're already doing that, you might try running "plutil -lint" on 
each of the .plist files in your modified extension to make sure they're not 
suffering from a syntax error.

Also, you are using "--distribution" when you build your .xar, right?  
According to @18 above Safari can get quite disturbed if any extended 
attributes are included.  Safari 5.0.4 was released 2 months after @18 was 
reported and 5.0.5 since that (as well as some other system updates), so it's 
possible that's not an issue anymore.

Original comment by mack...@gmail.com on 23 Jun 2011 at 2:00

[GoogleCodeExporter]

I'm doing as you recommend.

In comment 25

In step 2. I extract the certificates of a valid extension built with Safari. 
(someExtension.safariextz)

Step 6. can be done just one time.

Steps 7., 8. and 9. should be executed on the new xar file (yes I'm using 
--distribution when I create the xar), every time that some file change. 
MyExt.safariextz)

I wasn't aware of plutil so I try it on my Info.list
$ plutil -lint Info.list
Info.list: OK

Thanks for your help, it has been pretty useful.

Original comment by herberth...@gmail.com on 23 Jun 2011 at 3:59

[GoogleCodeExporter]

Hi 

I successfully create a new extension with the certs from another extension. 
The only restriction was that I have to rename the folder of my code to the 
folder name that was used in the original extension.

I was able to determine that because the extension from where I was extracting 
the certs is an outdated version of my code. And the folders had different 
names.

I don't know if this is something expected, I hope not, because is a little 
annoying, but easy to fix in a script.

Is this code going to be part of a normal release of XAR?

Original comment by herberth...@gmail.com on 23 Jun 2011 at 6:20

[GoogleCodeExporter]

Hi

I search any way for automatic create my extension. Can any help me in download 
patch from this Issue. If need, my mail - stskilur@gmail.com

sorry, my poor english.

Original comment by ser...@oneclickdev.com on 9 Sep 2011 at 1:57

[GoogleCodeExporter]

I, as well, am trying to create a Safari extension on the command line. I have 
tried the steps above but have not yet had success. My attempts have been on 
Windows Vista.

XAR is being build with Cygwin. XAR was downloaded from: 
http://xar.googlecode.com/issues/attachment?aid=-2625211052210458594&name=xar_23
2v3_src.tar.gz&token=cec294aa52575bb67df387fb6e6c3799 which is the TGZ file in 
comment #17.

I am starting with a Code Signing Certificate from GoDaddy. Once you buy a 
certificate on GoDaddy a download link is provided. Using FireFox, when you 
click download the public/private key exchange just happens and the certificate 
can be found in Tools -> Options -> Advanced -> View Certificates. From here 
the cert can be exported to a .p12. This is where I start...

I have a directory MyExt.safariextension that can be loaded into Safari's 
Extension Builder and saved to MyExt.safariextz that will install.

1. gunzip xar_232v3_src.tar.gz ; tar xf xar_232v3_src.tar
2. cd xar_232v3_src ; ./configure ; make install
3. xar --distribution -c -f MyExt.safariextz MyExt.safariextension
4. openssl pkcs12 -in key.p12 -out key.pem -nodes
5. openssl x509 -inform PEM -in key.pem -out key.der -outform DER
6. : | openssl dgst -sign key.pem -binary | wc -c > siglen.txt
7. xar --sign -f MyExt.safariextz --data-to-sign sha1_hash.dat --sig-size `cat 
siglen.txt` --cert-loc key.der
8. (echo "3021300906052B0E03021A05000414" | xxd -r -p; cat sha1_hash.dat) | 
openssl rsautl -sign -inkey key.pem > signature.dat
9. xar --inject-sig signature.dat -f MyExt.safariextz
10. Launch Safari and open MyExt.safariextz -- This generates a Cannot install 
error
No step (other than 10) generates an error.

Can anyone point out an error in the above steps? 
Did I miss a step?
Any suggestions as to what I should try next?

I will post my solution, step by step, with the src bundle (if changes are 
necessary) upon finding a proper solution.

Original comment by mah...@gmail.com on 27 Oct 2011 at 12:27

[GoogleCodeExporter]

mah...@gmail.com,

Perhaps you did not see the text "Safari appears to be hard-coded to only 
accept extensions signed by Apple-issued certificates" in comment #17.  A 
GoDaddy issued certificate almost certainly will not work no matter what kind 
it is (and a plain Code Signing certificate is not the same as a Safari 
Extension Signing certificate anyway, see 
http://images.apple.com/certificateauthority/pdf/Apple_WWDR_CPS_v1.7.pdf).

I don't believe Safari will ever accept an extension signed by a 
non-Apple-issued certificate.

Original comment by mack...@gmail.com on 27 Oct 2011 at 12:56

[GoogleCodeExporter]

I tried the version listed in comment #17, and I get segfaults when I try to 
use it.  Let me know what other information you need.  I am running OSX 10.6.8.

Original comment by jonathan...@gmail.com on 11 Nov 2011 at 10:29

[GoogleCodeExporter]

Apologies in advance, but I don't seem to see xar_cli_signature_232.patch 
attached anywhere, so I never seem to get the --sign flag added to xar.  Does 
anyone have it, or am I way off base here?

Original comment by bbitti...@gmail.com on 7 Feb 2012 at 8:00

[GoogleCodeExporter]

Also getting segmenation faults:

xar version:  xar 1.6dev_sigv3

macosx 10.7.2

Original comment by bbitti...@gmail.com on 7 Feb 2012 at 9:13

[GoogleCodeExporter]

The segmentation fault in question:

(gdb) 
#0  0x00000001000e87e1 in EVP_PKEY_CTX_free ()
#1  0x00000001000dc4ea in EVP_MD_CTX_cleanup ()
#2  0x00000001000dc852 in EVP_DigestFinal ()
#3  0x0000000100012d17 in xar_close (x=0x100500b30) at archive.c:579
#4  0x0000000100004469 in archive (filename=0x7fff5fbffaaa "test.safariextz", 
arglen=1, args=0x100500af0) at xar.c:972
#5  0x00000001000085d4 in main (argc=6, argv=0x7fff5fbff948) at xar.c:2049
(gdb)

Original comment by bbitti...@gmail.com on 7 Feb 2012 at 9:54

[GoogleCodeExporter]

hi,
i have also tried building safari extension using xar and able to get success 
to some point but when it creates the safarixtz then it crashes in segmentation 
fault.

does anyone able to solve this segmentation fault?

rstanwar

Original comment by ram.rsta...@gmail.com on 24 Feb 2012 at 8:00

[GoogleCodeExporter]

hi,

can anyone send the updated version of xar with this segmentation fault fix.

rstanwar

Original comment by ram.rsta...@gmail.com on 24 Feb 2012 at 11:19

[GoogleCodeExporter]

Hey,

We on Crossrider also looking for this functionality.
Would like to get some help with that issue.

Thanks,
Shay.

Original comment by dad...@crossrider.com on 3 Jul 2012 at 1:14

[GoogleCodeExporter]

I have suceeded in it. I was successful to build safariextz package from 
xar.exe and safari can read it. I have had to do some changes in sources 
(xar.c, stat.c, signature.c, hash.c, look for "// SRAJER START" string in 
sources).

I built it in cygwin on windows 7.

I am attaching sources, my batch script for windows and 2 Apple certificates.
rsadmin.der and rsadmin.pem are public and private keys.

Note that command

(echo "3021300906052B0E03021A05000414" | xxd -r -p; cat sha1_hash.dat) |
  openssl rsautl -sign -inkey key.pem > signature.dat

is simply replaced by

%openssl_path% rsautl -sign -inkey %certs_path%\rsadmin.pem -in sha1_hash.dat 
-out signature.dat

sha1_hash.dat file already includes binary sha1 prefix (it is hardcoded in 
xar.c:signingCallback function).

Bye

Original comment by roman.sr...@gmail.com on 19 Jul 2012 at 9:05

Attachments:

• xar.zip
• apple1.pem
• apple2.pem
• build.cmd

[GoogleCodeExporter]

The segfaults mentioned above (comment 32, comment 34, comment 35, comment 36, 
comment 37) are probably all caused by the same issue.  The segfault stack 
trace from comment 35 in particular is caused by compiling xar against an older 
version of OpenSSL (libcrypto) but linking it against a newer version.

If you have, for example, MacPorts installed and in your path and MacPorts has 
built its own libxml2 and a newer OpenSSL then the bizarre OS X linker behavior 
will cause this to happen.

Only the very latest sources support compiling against an older OpenSSL but 
linking with a newer one as special precautions must be taken to avoid memory 
overruns and crashes in this case.

Changes inspired by the code modifications from comment 39 have been added.  
Specifically a new --digestinfo-to-sign option that outputs the data to be 
signed with the proper DigestInfo header already prepended.

The signature-creation-time element value now gets written with tenths of a 
second.

The message digest name written as the extracted-checksum/archived-checksum 
style attribute value now matches the case of the value written for the toc 
checksum message digest name.  This is purely cosmetic as both UPPER and lower 
case work and are recognized by all existing xar versions.

Building and running on cygwin should also be improved.

Many other bug fixes and enhancements have been added as well.

As these patch files have been getting cumbersome I have imported everything 
(all the current subversion history) into a git repository on GitHub and then 
checked in the command line signature additions on top.  Pre-bootstrapped 
tarball downloads are available on GitHub as well as more documentation on 
signing, other enhancements, etc.

The project page (with many helpful links) is at:

  http://mackyle.github.com/xar

Pre-bootstrapped tarballs of the latest version can be downloaded from:

  https://github.com/mackyle/xar/downloads

And finally the git sources can be cloned from any of these:

  git://github.com/mackyle/xar.git
  http://github.com/mackyle/xar.git
  https://github.com/mackyle/xar.git

Any further problems with the command line signature support please create an 
issue for the GitHub project.

Original comment by mack...@gmail.com on 18 Sep 2012 at 5:04

[GoogleCodeExporter]

Hi Kyle,

What's the latest way to produce a signed, installable Safari extension 
package? What format should the certificate be? What are the commands?

Original comment by amiag...@ghostery.com on 1 Oct 2012 at 10:43

[GoogleCodeExporter]

Please see the project page at:

http://mackyle.github.com/xar

And then in the More Info section click on the "How to sign an xar archive" 
link.  (The xar man page is also available via the "xar man page" link in the 
same section.)

Original comment by mack...@gmail.com on 3 Oct 2012 at 11:36

[GoogleCodeExporter]

Ah, thanks. I missed it going through the first time around.

The "how to sign" page has all the necessary information (how to extract the 
certificate chain, how to convert p12  to pem, ...) but it's a little jumbled 
when you just want to package Safari extensions from the cli. It's not clear 
that you need to read all of it, but you do. So maybe a note to that effect on 
the homepage is warranted.

Thank you for the patches and the documentation.

Original comment by amiag...@ghostery.com on 5 Oct 2012 at 6:02