MacOS (PIV.tokend) requires Card Capability Container file

[Davidllo]

Hi,

I have been debugging "PIV.tokend" from the "Smartcard Services for Macintosh" add on for MacOS (the standard Apple smart card package), and it is failing as it is trying to use the unique identifier in the CCC file. This is technically a mandatory file, like the CHUID.

I think that PIV tool should have an "-action initialize" option that installs a CHUID and CCC (at least - there are other mandatory files that probably be considered).

The "-action status" option should probably flag missing mandatory files as errors. Missing CHUID won't work on Windows, missing CCC won't work on Mac.

There is a technical discussion here: https://lists.macosforge.org/pipermail/smartcardservices-users/2015-October/000539.html

I can produce some patches if needed.

[klali]

So.. the mandatory objects (specified in sp80073-4 part 1) are: Card Capability Container Card Holder Unique Identifier X.509 Certificate for PIV Authentication X.509 Certificate for Card Authentication Cardholder Fingerprints Cardholder Facial Image Security Object

That's quite a handful. I'm not opposed to adding writing of more objects to yubico-piv-tool if that is useful for people. One interesting point about the CCC is that the PIV specs seem to allow a very minimal CCC: f0 00 f1 00 f2 00 f3 00 f4 00 f5 01 10 f6 00 f7 00 fa 00 fb 00 fc 00 fd 00 fe 00 being valid as I read it. So an interesting question here is what it should look like to be useful for most cases?

[Davidllo]

In general, I prefer to justify not implementing bits of spec, so:

• Fingerprints and Facial Image require additional user input, so you can't have these by default. (*)
• The signature on the CHUID will be a pain for newbies, so a valid signature needs to be optional.
• The Security Object also requires a signature too.

My reading of the Apple code is that they use the 21 byte card id from the CCC as a card identifier, so you could probably fill F0 with 21 bytes of random as you do for the CHUID. Even a completely empty CCC will fix the problem though (not tried it, though).

(*) Note that NIST do actually provide FOSS tools and a "match-on-card" Javacard applet for facial/iris/fingerprint support if you want to play with them (http://www.nist.gov/itl/iad/ig/nigos.cfm).

[mouse07410]

One interesting point about the CCC is that the PIV specs seem to allow a very minimal CCC: f0 00 f1 00 f2 00 f3 00 f4 00 f5 01 10 f6 00 f7 00 fa 00 fb 00 fc 00 fd 00 fe 00 being valid as I read it.

I don't think the above is valid, because I don't think that F0, F1, and F2 can be empty.

Regardless, the following appears to be a good working example of a minimalistic CCC defining a PIV card:

5344f015a0000001164b021764d935c787c5e6a2d7331643d2f10121f20121f300f40111f50110f6110000000000000000000000000000000000f700fa00fb00fc00fd00fe00

Most of its fields can reasonably be zeroes (Fx 00), but some (F0 in particular) should better be filled with something reasonable. One reason - it's F0 that defines the card as PIV or CAC (or neither, in which case some apps would and some would not continue talking to it).

I'm not sure whether we want to put empty objects for Cardholder Fingerprints and Facial Image, or it could be OK to just omit them altogether. Since NEO has a different form-factor than CAC or PIV identity cards - I'm not sure it makes sense for us or Yubico to actually put either of these two fields on it automatically. Perhaps it's something a user (or his IT) can do themselves if that's what is needed (and piv-tool would handle the job nicely :).

I haven't looked at the Security Object - perhaps it should be there, and maybe even non-empty?

So far it is certain that CCC and CHUID must be present. It also makes sense in CCC to specify what versions of the specs the card implements (F1 and F2) - otherwise I suspect enough of the apps out there would turn their noses away form a card that claims it supports specs v0... But the CCC example above should be good enough to copy-cat from. :-)

[Davidllo]

Hi Mouse,

The document (http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-73-4.pdf) does have in section 3.1.1:

".... Therefore, all mandatory data elements of the CCC, except for the data model number, may optionally have a length value set to zero bytes (i.e., no value field will be supplied). Unused optional data elements shall be absent..."

BUT - you seem to know a lot more about this than I do, so I'm not going to quibble! My only desire is for people to log into Windows without the three days I spent debugging CSP container names (CHUID), and for people to plug into a Mac, install the Smart Card Service .pkg that Apple's website recommends, and have KeyChain recognise the device without spending two days trying to compile and debug PIV.tokend (CCC)...

I promised a patch, so I copy&pasted the CHUID code in PIV tool and populated 21 bytes of CCC with random stuff. Mac now works out of the box (for me)...

/* Card Capability Container. All mandatory entries are NULL as allowed by the

• specifications, except for the the 21 byte card identifier (F0) and the
• Registered Data Model Number (F5) that must be 0x10 */ unsigned const char card_capability_tmpl[] = { 0xf0, 0x15, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xf1, 0x00, 0xf2, 0x00, 0xf3, 0x00, 0xf4, 0x00, 0xf5, 0x01, 0x10, 0xf6, 0x00, 0xf7, 0x00, 0xfa, 0x00, 0xfb, 0x00, 0xfc, 0x00, 0xfd, 0x00, 0xfe, 0x00 };

  define CCC_CARD_IDENTIFIER_OFFS 2

static bool set_ccc(ykpiv_state *state, int verbose) { unsigned char ccc[sizeof(card_capability_tmpl)]; ykpiv_rc res;

memcpy(ccc, card_capability_tmpl, sizeof(ccc)); if(RAND_pseudo_bytes(ccc + CCC_CARD_IDENTIFIER_OFFS, 0x15) == -1) { fprintf(stderr, "error: no randomness.\n"); return false; } if(verbose) { fprintf(stderr, "Setting the Card Capability Container to: "); dump_hex(ccc, sizeof(ccc), stderr, true); } if((res = ykpiv_save_object(state, YKPIV_OBJ_CAPABILITY, ccc, sizeof(ccc))) != YKPIV_OK) { fprintf(stderr, "Failed communicating with device: %s\n", ykpiv_strerror(res)); return false; }

return true; }

[mouse07410]

A few off-hand & brief comments.

".... Therefore, all mandatory data elements of the CCC, except for the data model number, may optionally have a length value set to zero bytes (i.e., no value field will be supplied). Unused optional data elements shall be absent..."

In theory that's correct. However, just like not all the PIV cards are 100% compatible with NIST 800-73-{3,4}, similarly not all the applications are 100% compatible with that document. Both cards and apps make assumptions.

So it would be prudent to provide some "low-hanging" data for such apps to "hang on to".

BUT - you seem to know a lot more about this than I do, so I'm not going to quibble!

:-)

My only desire is for people to log into Windows without the three days I spent debugging CSP container names (CHUID), and for people to plug into a Mac, install the Smart Card Service .pkg that Apple's website recommends, and have KeyChain recognise the device without spending two days trying to compile and debug PIV.tokend (CCC)...

A noble desire, but again - I respectfully disagree (wrt. OS X - I know nothing and couldn't care less about Windows). In my experience it is not enough to have Keychain Access to recognize the device. Because in my case, just adding CCC to a standard NEO that had only all the keys provisioned to it (and CHUID generated by yubico-piv-tool) was sufficient to get 100% reliable Keychain Access cooperation. However it was not enough for Apple Mail.

And I don't need Keychain Access for itself - merely as a sign that there's a good chance for other system apps (like Safari and Apple Mail, which is what I really care about) to be able to work with this card.

I copy&pasted the CHUID code in PIV tool and populated 21 bytes of CCC with random stuff.

May I suggest that random stuff is not the best for F0 field of CCC? And we don't need to cut this corner, since we have pretty good idea what should go there. For example, see CCC that I posted.

Mac now works out of the box (for me)...

"Mac" what? Keychain, or Apple Mail, or...? There's a big difference.

[Davidllo]

Hi,

I must confess that my requirements are limited to "appears in keychain/certificate store", "log in to Windows" and TLS client authentication in Safari, Firefox, and Internet Explorer on all Mac, Windows and Linux. I haven't tested with CMS.

The CCC structure is documented in section 6.3.1 of http://csrc.nist.gov/publications/nistir/nistir-6887.pdf

Your bit string parses to...

f0:a0000001164b021764d935c787c5e6a2d7331643d2 GSC-RID = a000000116 [GSC-IS Data Model] ManufacturerId = 4b [DMDC Card Technologies - ??? Couldn't find a list] CardType = 02 [javaCard] CardId = 1764d935c787c5e6a2d7331643d2 [???Random???] f1: 21 CCC Version number 2.1 f2: 21 Capability Grammar Version number 2.1 f4: 1 PKCS#15 compatible card f5: 10 Data Model number 16 [Mandatory] f6: 0000000000000000000000000000000000 Access Control rule table [I'm not sure what this is doing]

In any case, my patch can be updated in the obvious way to put a structure like this on a Yubikey...

Regards!

[mouse07410]

Your bit string parses to...

f0:a0000001164b021764d935c787c5e6a2d7331643d2

Yeah...

GSC-RID = a000000116 [GSC-IS Data Model]

Above is mandatory, and please keep as is.

ManufacturerId = 4b [DMDC Card Technologies - ??? Couldn't find a list] - random, can change - but try to avoid some known ones so there is no consequences of assuming a wrong manufacturer.

CardType = 02 [javaCard] - please keep as-is.

CardId = 1764d935c787c5e6a2d7331643d2 [???Random???] - you got it - 14 random bytes

f1: 21
CCC Version number 2.1
f2: 21
Capability Grammar Version number 2.1
f4: 1
PKCS#15 compatible card
f5: 10
Data Model number 16 [Mandatory]

Above is mandatory - please keep as is.

f6: 0000000000000000000000000000000000

Access Control rule table [I'm not sure what this is doing] Yes. I don't know for sure, but recommend keeping this as is.

I must confess that my requirements are limited to "appears in keychain/certificate store", "log in to Windows" and TLS client authentication in Safari, Firefox, and Internet Explorer on all Mac, Windows and Linux. I haven't tested with CMS.

Are you saying that you've tested and were able to use TLS client authentication (with keys stored in NEO) with Safari and Firefox on Mac? If so, I'd like some details please.

[Davidllo]

Hi,

Here is the template with the data Mouse recommends (note that there is now only 0xe bytes of RNG).

/ Card Capability Container. The Manufacturer ID is 0xff, there are 14 random bytes in the CardId / unsigned const char card_capability_tmpl[] = { 0xf0, 0x15, 0xa0, 0x00, 0x00, 0x01, 0x16, 0xff, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xf1, 0x01, 0x21, 0xf2, 0x01, 0x21, 0xf3, 0x00, 0xf4, 0x01, 0x01, 0xf5, 0x01, 0x10, 0xf6, 0x11, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xf7, 0x00, 0xfa, 0x00, 0xfb, 0x00, 0xfc, 0x00, 0xfd, 0x00, 0xfe, 0x00 };

define CCC_CARD_IDENTIFIER_OFFS 9

I can't go into too much detail about what I am using the Neo for. My environment is:

• IIS on Win2k12R2 with mandatory Client Auth
• Certificate from MSCA Win2k12R12 Domain Controller (SmartcardUser template)
• Configuration from the Linux PIV tool
• RSA 2048 bit keys
• Yubikey: 0x1050 0x0112
• I have switched the Yubikey into pure PIV mode (accidentally) so I can't read the firmware version
• Mac OSX: El-Captain. Smart card Services v2.1.1 with PIV.tokend for Safari 9.0.
• Mac OSX: PKCS11 from OpenSC for Firefox
• Windows: Microsoft PIV drivers Win2k12R2 (and OpenSC PKCS11 for Firefox)
• Linux: Fedora with stock OpenSC/PCSCLite

Mostly it "just worked" for me by compiling from source. I am expecting to get this thing to work everywhere.

This bug should be the last piece of the MacOSX puzzle. I do need the --action=initialize feature though (other people will be doing the enrolment).

[mouse07410]

I can't go into too much detail about what I am using the Neo for.

No need to. :-)

I have switched the Yubikey into pure PIV mode (accidentally) so I can't read the firmware version

Not important, but Yubico released a nice GUI tool Yubikey NEO Manager, which could help you here.

I do need the --action=initialize feature though (other people will be doing the enrolment).

I concur.

@klali , it would be great if yubico-piv-tool would generate and write to the NEO token CHUID (which it already does) and CCC (filling those 14 bytes of CardId with random data) upon request.

[mouse07410]

@Davidllo could you please tell me how to download the minimal SmartCardServices source to build a usable PIV.tokend? I'd like to play with it a little.

[Davidllo]

The installers for Smart card services (signed and maintained by Apple employees) are here http://smartcardservices.macosforge.org/trac/wiki/installers. There is build documentation: http://smartcardservices.macosforge.org/trac/wiki/documentation/building

The source is here: svn checkout https://svn.macosforge.org/repository/smartcardservices

I didn't have any luck with DarwinBuild, so I went straight to Xcode. There are some complexities - the biggest one is to use glibc rather than clang. Also there are missing frameworks that get pulled from OpenSC, and the build system seems to be for Yosemite and needed rework. It took me a day or so to compile it (then all of 15 minutes to debug the Yubikey).

In any case, with the CCC patch here building a (more) valid PIV card, you won't need to modify the PIV.tokend code at all. Note that OpenSC doesn't actually install on ElCapitain anymore (although it still works with Firefox) they need to update the installers. Like a lot of people on the Internet, I couldn't get OpenSC's tokend driver to work with ElC or Yosemite - I didn't have to as Apple supply PIV drivers that work with the NEO.

[mouse07410]

Thank you!

My concern here is the problem with Apple Mail (and maybe other applications, or Keychain API) that seem to get in trouble with tokend regarding the status of the token (unlocked, or not). It looks like Apple Mail (or Keychain API, whatever) "thinks" that the token is still unlocked - while the token expects the PIN again, and thus fails the operation.

I haven't figured out yet whether it is something that can be addressed by modifications to F6 field of CCC, or requires deeper surgery, maybe in the bowels of PIV.tokend...

Like a lot of people on the Internet, I couldn't get OpenSC's tokend driver to work with ElC or Yosemite

Join the club! :-) :-(

I didn't have to as Apple supply PIV drivers that work with the NEO.

That's what I hope for, with PIV.tokend, PKard.tokend, and maybe Centrify Express...

[Davidllo]

This probably isn't the right place to discuss smart card services... If you are who I think you are, then I don't mind doing a debug session. I'll put together some more detailed notes on compiling PIV.tokend and post them to the Smart Card Users mailing list.

[mouse07410]

Saw your post on SmartCardServices, thank you! Will do some experiments, and let you know (and probably ask for some help ;).

I'm trying to figure out why:

a. Once in a while tokend would get crazy and stop recognizing the card - when this happens, the log shows that the card replies with 6a 82 when tokend tries to retrieve CCC (which I know is there because I can retrieve it manually via OpenSC tools) . Here's an example failed session (tokend refused to even deal with the card - and this is with the card it worked happily for a few days):

2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSGSC] PKard starting: 1.6.1, Copyright © 2003-2014 Thursby Software Systems, Inc.
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), tsstoken] TSSToken ctor
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), tokend] _probe: begin
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), tsstoken] TSSToken::probe - card protocol=2
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), tsstoken] TSSToken::probe Yubico Yubikey NEO OTP+U2F+CCID atr=3bfc1300008131fe15597562696b65794e454f7233e1
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), probe] default score=0, flags = 0
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardClass] presence NOT detected '/var/db/TokenCache/tokens/com.thursby.caching.disabled' uid=91
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardClass] presence NOT detected '/Library/Logs/com.thursby.pki.caching.disabled' uid=91
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardCAC] ctor
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardCACNG] ctor
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardCACNG] probe
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSApplicationDataObject] CCC: mACR=0
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), APDU] 00a4040007a000000116db00->6a82
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardCACNG] Get data failed
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSApplicationDataObject] CCC not present on card.
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), CACNG] probe: not presentOnCard
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), tsstoken] probe TSSCardCACNG score=0
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardClass] presence NOT detected '/var/db/TokenCache/tokens/com.thursby.caching.disabled' uid=91
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardClass] presence NOT detected '/Library/Logs/com.thursby.pki.caching.disabled' uid=91
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardPIV] ctor
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardPIV] probe
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSApplicationDataObject] CCC: mACR=0
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardClass] Cache miss for CCC
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSPIVOperations] selectContainer NOT called
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), APDU] 00cb3fff055c035fc107->6d00
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), tsstokend] exchangeAPDU result 6D00
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), tsstoken] probe TSSCardPIV score=0
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardClass] presence NOT detected '/var/db/TokenCache/tokens/com.thursby.caching.disabled' uid=91
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardClass] presence NOT detected '/Library/Logs/com.thursby.pki.caching.disabled' uid=91
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardCAC] ctor
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardCAC] probe
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardCAC] probe: selecting cert_signing_app
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardClass] select BEGIN mCurrentApplet=0x0 applet=0x7fa08850e0a0
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), APDU] 00a4040007a000000079010102->6a82
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), tsstoken] select END [FAILURE 6A 82]
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), TSSCardCAC] probe: something threw, probably not a CAC
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), tsstoken] probe TSSCardCAC score=0
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), tsstoken] probe() = 0
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), tokend] _probe: end flags=0 returning score=0  uid=''
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), tssgsc] tssgsc_terminate due to SIGTERM
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), tsstoken] TSSToken::terminate(15,0)
2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), tokend] TSSToken::terminate: writing writing name '' to cached PrintName file: '/../PrintName' ...

2015-10-30 00:20:18 +0000 [77826 (0x7fff70c88300), tokend] TSSToken::terminate: error 13 'UNIX error exception' writing 1 bytes to cache file: /../PrintName

b. A component of tokend framework that is supposed to keep track of the token status (need PIN, PIN cached, etc) fails to do so, or gets confused along the way. I've seen it with all the tokends I tried. No idea whose fault it is - though if the card occasionally throws 6a 82 or 6d 00 when it isn't supposed to, it is at least partially to blame.

I'd appreciate if you could test S/MIME for me - just send a bunch of digitally signed emails to anywhere (your other account, your colleague who doesn't mind, me), preferably using Apple Mail. Of course, signing with the Digital Signature key on the NEO token. And let me know if it works smoothly, or if you observe any problem.

[Davidllo]

Bad (?) news I'm afraid... Apple Mail S/MIME "just works" for me with the Yubikey Neo and PIV.tokend, so I can't really debug your issue.

It prompts for the PIN quite a lot - I guess that's the point of slot 9c course! Both decryption and signing worked without any hassle - quite a nifty integration actually. All I needed was to have the correct email address in the certificate, and plug in the Yuibikey.

Once this bug is fixed, everyone else will be able to experience PIV joy on a Mac, too!!!

[mouse07410]

Bad (?) news I'm afraid... Apple Mail S/MIME "just works" for me with the Yubikey Neo and PIV.tokend, so I can't really debug your issue.

Wow! So for you Apple Mail sends several signed emails in a row, prompting you for your PIN every time? That's what I'd like to see!

You run on Yosemite, or on El Capitan?

Also, I confess I'm having trouble getting SmartCardServices Tokend to build. I added all the External Frameworks necessary - but the compiler cannot find header files from them...

[Davidllo]

This is El Capitan. I've sent 10 emails in a row, each with a PIN prompt. It looks ok...

You can cheat with the header files if Xcode isn't playing right - just symlink the various directories into an "include" directory.

FWIW -- The Apple experts here say that the SMIME support in the Mail app has gotten a little flakey recently in general. It's not related to particular smart cards - usually it is the public key encryption side that goes wrong.

[mouse07410]

This is El Capitan. I've sent 10 emails in a row, each with a PIN prompt. It looks ok...

Then you don't experience the problem I'm having. Good...

You can cheat with the header files if Xcode isn't playing right - just symlink the various directories into an "include" directory.

I must admit that I'm not comfortable with the entire Xcode build environment and frameworks - longing for the old-fashioned Makefile. :-) I suspect I'm not doing something right with the Project, rather than Xcode not playing right... :-(

Would appreciate some help, if feasible.

Oh, and one more thing: would you be able to test your NEO (with the CCC that we discussed in this thread written to it) with the stock PIV.tokend (i.e. not the one you compiled from source, but one from the binary installer package), either from SmartCardServices, or from Centrify Express?

Thanks!

[Davidllo]

would you be able to test your NEO (with the CCC that we discussed in this thread written to it) with the stock PIV.tokend

Yes - the stock version works fine. There are two possible fixes: change PIV.tokend to allow cards to not have a CCC entry, or fix this bug to put the CCC entry on Yubikeys.

Given that klali willingly admits that CCC is a mandatory object, I guess this is the correct bug to fix.

FWIW -- your traces reference Thursby, which isn't part of OpenSC or PIV.tokend. I guess that's where your problem is. Also, you should test with ElCapitan if you haven't done so already. If there's a fix in Apple Mail, then there isn't much point trying to debug PIV.tokend!

[mouse07410]

Yes - the stock version works fine.

Excellent.

Given that @klali willingly admits that CCC is a mandatory object, I guess this is the correct bug to fix.

I concur 100%.

FWIW -- your traces reference Thursby, which isn't part of OpenSC or PIV.tokend.

Because that's the only tokend that provides decent traces. And it helped immensely to figure out what seemed to be wrong with the NEO card (why tokend was refusing to work with it), and how to remedy the problem. And also to confirm the fix.

I guess that's where your problem is.

???

I'm pretty sure the above statement is 100% wrong.

After NEO was supplied with CCC (and its contents was a combination of what the NIST standard said must be there, what the GCS-IS document described, and what Thursby example showed) and Thursby PKard.tokend started working with it - PIV.tokend started working with it too. And after one bug in OpenSC.tokend was fixed (that fix still isn't merged), OpenSC.tokend too started working with NEO (and CAC, if it matters).

So now I have most of the tokends working with NEO. :-)

Also, you should test with El Capitan if you haven't done so already.

I need osxfuse to work more than I need perfection in tokend or Apple Mail. So until I am certain osxfuse and EncFS work fine on El Capitan, I cannot move to that version. And the only way to "test that version" is to upgrade from Yosemite to El Capitan (I don't have spare machines to run test OS).

Also, FWIW, people complain about Apple Mail in El Capitan even more than on the older versions.

[mouse07410]

@klali could you tell what it would take for you to add CCC creation to yubico-piv-tool? Most of the format/content of it is pre-defined, the only changing part is 14 random bytes... And you already generate and write to the card CHUID, so you have the template for doing that...?

[mouse07410]

@Davidllo I must say that after upgrading to El Capitan, I sent a few RSA-signed emails fine using MS Outlook 2011 and Thunderbird, but not even one successfully signed email via Apple Mail.

[Davidllo]

I don't have much experience with Apple Mail or OSX - a few other people here work with that OSX and those features of cards.

Let's start with verifying the basics: are you using the latest "Smartcard Services for Mac" PIV.tokend stuff as recommended by Apple? ElC and Apple mail 9.1. Check that you have run the set-chid and set-ccc (we still need to fully finish running a test cycle with that change) options with this tool. Do you see the Yubikey certificates appear in KeyChain?

Thunderbird uses PKCS11 so you are probably using OpenSC for that one at least. I know that there are/were issues with opensc.tokend on Mac, and it isn't what Apple recommend so we don't use that (but we do use PKCS11 libraries from the OpenSC project).

When Apple Mail signs or decrypt emails, all it does is look through KeyChain for certificates with appropriate email addresses. If it finds appropriate certificates it will check the extensions and show a button allowing you to sign the email. To test your certificate extensions, simply install one as a software certificate in KeyChain.

I am assuming that you see the Apple Mail button, but the signing step doesn't work. There really isn't anything special happening at this point - either the RSA sign works or it doesn't... I would write a test program that performs signing, and also check other KeyChain-based programs (Safari TLS client authentication being the obvious one). This would finger Apple Mail or the middleware stack.

So next up, which slots on the PIV card are you using? Technically you should be using the document signing certificate slot (although I don't think that PIV.tokend enforces that). For this, if you can, you should also test with the NIST PIV test cards http://csrc.nist.gov/groups/SNS/piv/testcards.html. This will isolate problems specific to the Yubikey, as opposed to a software stack problem.

We have other tools that we use to dump out APDUs and parse them, so in the worst case we'd next try to break down to protocol errors with the YubiKey, but other than it (previously) responding to a request for CHUID/CCC with "file not found", we haven't had issues with what PIV.tokend tries to do (which isn't very much, TBH). From my (limited) understanding of the MacOS stack, though, I doubt that your problem is with the Yubikey or the PIV protocol - it's probably an OS change to the interfaces between tokend and KeyChain...

[mouse07410]

Let me start with a comment. It seems that the problem is with how Apple (now?) is interpreting certificate extensions. Playing with those (mostly - email in DN, and RFC 822 in SAN), I was able to get back to situation I had in Yosemite - messages get signed or not depending on the moon phase.

Let's start with verifying the basics: are you using the latest "Smartcard Services for Mac" PIV.tokend stuff as recommended by Apple?

Yes, and it seems to behave the same way as PKard.tokend and OpenSC.tokend (after a healthy dose of treatment :). Maybe a little more stable, and I really prefer its way of identifying the token in the Keychain via CN of the first cert rather than as "PIV-II" (as OpenSC.tokend does), but other than that I haven't noticed any difference.

ElC and Apple mail 9.1. Check that you have run the set-chid and set-ccc (we still need to fully finish running a test cycle with that change) options with this tool. Do you see the Yubikey certificates appear in KeyChain?

Yes I do. That part seems to work flawlessly. Though I must add that depending on the combination of SAN content, Apple (Keychain Access, and I assume Mail) may or may not recognize that a given certificate is bound to a given email address (and yes, everything there is in lowercase, so no ambiguity). See below.

Thunderbird uses PKCS11 so you are probably using OpenSC for that one at least.

Yes, absolutely. At the latest/current level of patches. And Thunderbird seems the closest/best consistency-wise, with the best success ratio. MS Outlook is its close second, until you factor ECDSA in. And with Hotmail (which I use to get these tests done), I'm getting that ugly mismatch between signing and sending email addresses (no matter what I do on the sending end, it seems that the email appears sent by (at)hotmail.com... Oh well...

I know that there are/were issues with opensc.tokend on Mac, and it isn't what Apple recommend so we don't use that (but we do use PKCS11 libraries from the OpenSC project).

Well, it so happens that OpenSC.tokend is the only one that I both have the source code of, and can successfully build on my boxes. After I started working on it, and with the help of Doug Engert and Frank Morgner, I'd say OpenSC.tokend does as good a job with RSA keys as any, and does ECDSA reasonably well. I could not check ECDH because it doesn't look like there is any application that uses tokend and supports ECDH in S/MIME. Definitely not Apple Mail, and not MS Outlook.

When Apple Mail signs or decrypt emails, all it does is look through KeyChain for certificates with appropriate email addresses. If it finds appropriate certificates it will check the extensions and show a button allowing you to sign the email.

We (it is probably quite typical) use two key pairs - one for signing, and another one for encrypting.

With El Capitan I'm having a really hard time with the SAN extension. I guess my specific needs contribute to this. I need one certificate to support more than one email address. Using Hotmail as an example, I want each key pair to support both mouse07410@hotmail.com and mouse07410@outlook.com: Here's the DN (common for both signing and encrypting certs):

Extensions of the signing cert:

Extensions of the encrypting cert:

With the above templates, when I ask Keychain Access to look for mouse07410@outlook.com, it gives me both encrypting and signing certs. But when I ask for mouse07410@hotmail.com, it gives me only the signing certificate...

To test your certificate extensions, simply install one as a software certificate in KeyChain.

I did, and got quite a shock - see above.

I am assuming that you see the Apple Mail button, but the signing step doesn't work.

No consistency here. Sometimes it gives the "normal" check-mark and signs successfully. Sometimes it gives the "normal" check-mark, prompts for the PIN, but returns with Failure. Sometimes the check-mark has the check dark - in which case it may or may not successfully sign. And finally, a few times it just gave me that check-box grayed out (i.e., could not find the appropriate certificate). Drove me mad...

There really isn't anything special happening at this point - either the RSA sign works or it doesn't...

Oh, RSA works fine. I've tested it via command-line tools, using both OpenSSL and OpenSC (separately, and together - encrypt with one and decrypt with the other), and all is great. Interoperability there is beyond reproach. :)

I would write a test program that performs signing, ...

And encrypting with RSA. Perfectly fine. I even had fun writing and running them.

and also check other KeyChain-based programs (Safari TLS client authentication being the obvious one). This would finger Apple Mail or the middleware stack.

Yeah... This is a bit more hairy. Let me try this (the obvious issue here - I need to configure the other end :), and report.

So next up, which slots on the PIV card are you using?

Oh, I am using 9c (SIGN key) for signatures, and 9d (KEY MAN key) for encryption (or rather, decryption). No deviation here.

Technically you should be using the document signing certificate slot (although I don't think that PIV.tokend enforces that).

I don't know whether tokend enforces it, but want to do the right thing regardless.

For this, if you can, you should also test with the NIST PIV test cards...

Cost and time of the procurement process all but preclude me from doing that.

From my (limited) understanding of the MacOS stack, though, I doubt that your problem is with the Yubikey or the PIV protocol - it's probably an OS change to the interfaces between tokend and KeyChain...

I am 100% certain of that. Especially since the problems started to show up after the upgrade, with everything (configuration, certificate, tokend code) staying the same.

Do you see anything obviously wrong with the cert templates I posted above? Also, when I use Keychain's Certificate Assistant and ask it to evaluate this encryption certificate for S/MIME with email address mouse07410@outlook.com (same result for mouse07410@hotmail.com). DN is depicted above - it's the same for all of these experimental certs. I'm getting this:

Here are the extensions - do you see any offense there???

P.S. Signing cert always evaluates good:

[Davidllo]

Your KeyUsage extensions are definitely suspect. Check RFC 5280. It should include “digitalSignature”, it needs that for TLS Client Authentication as well. You even have dataEncipherment which I have never seen outside the spec - CMS always uses symmetric keys for bulk data encryption.

For CMS signing (see RFC 3850 section 4.4.2) I’d expect: digitalSignature + nonRepudiation For CMS encryption, I’d expect keyEncipherment

keyAgreement is online DH or ECDH, it wouldn’t use it for S/MIME. IIRC even ECDSS doesn’t use it.

I don’t where all the IPSec and Server Authentication stuff is coming from, though it isn’t a problem as you have the emailProtection usage, and the blanket AllKeyUsages in there.

You probably should take a look at your certificate generation (stick with soft-certificates while debugging them).

[Davidllo]

Here's a boring PDF that explains what extensions PIV/PIV-I expects: https://www.idmanagement.gov/sites/default/files/documents/pivi_certificate_crl_profile.pdf I turned it into a pretty trivial one page table showing the differences between them. Note that Worksheet 6: PIV-I Digital Signature Certificate Profile is an email signing certificate.

I think that it is fairly unusual to use separate signing and decryption certificates in practice. That may also be tripping up Apple Mail - certainly Apple Mail is quite enthusiastic in decrypting emails, and it may be "black holing" certificates that don't work every time...

[mouse07410]

Your KeyUsage extensions are definitely suspect.

For sure. But it looks like I showed an old experimental cert. :-(

Check RFC 5280. It should include “digitalSignature”, it needs that for TLS Client Authentication as well. You even have dataEncipherment which I have never seen outside the spec - CMS always uses symmetric keys for bulk data encryption.

Yeah... I put the old and the new encryption certs there, instead of signing and encrypting ones... Here's the correct screenshot of my signing cert extensions:

To be honest, I don't know what ECC algorithm is defined for S/MIME encryption. I assumed it's ECDH, but I can easily be wrong here, and I'm not aware of any application that I could use to find out what algorithm/protocol it actually requests for ECC-based S/MIME. :-(

I don’t where all the IPSec and Server Authentication stuff is coming from, though it isn’t a problem as you have the emailProtection usage, and the blanket AllKeyUsages in there.

Yeah... :-( Old stuff. Here's my current encryption extensions template:

You probably should take a look at your certificate generation (stick with soft-certificates while debugging them).

With that "boring PDF" that you sent, it just might work. :-)

I think that it is fairly unusual to use separate signing and decryption certificates in practice.

Not an option here - must stick with two separate ones. Where I think I am pushing it - it's with trying more than one RFC822 SAN in one cert. But I haven't heard yet from anybody whether it's "kosher" or not. :-(

That may also be tripping up Apple Mail - certainly Apple Mail is quite enthusiastic in decrypting emails, and it may be "black holing" certificates that don't work every time...

Grrr... Looks like I have to somehow get Apple involved...

[Davidllo]

Hi,

I can now confirm that set-ccc configures the Yubikey to work with "Smartcard Services" on El-Capitain.

Thanks!

[mouse07410]

@Davidllo at this time my certificates mostly work - except that when more than one RFC 822 SAN extension is added, Apple Mail (and Keychain Access) seem to respect only one of them. It also looks like OpenSC.tokend is pretty much usable now, after all the hard work. :-)

I seem unable to do the Xcode build of SmartCardServices. Asking Apple for help is, to put it mildly, useless. Would you be able to help me to get the build going?

[Davidllo]

Sure, looking back, I think that you were having problems getting Xcode to reference header files inside bundles. Is that correct?

One trick is to create a separate include directory and symlink them into the standard locations that C++ expects them to be. I am pretty sure that there's a way to get Xcode to work correctly though...