key tag calculation in ConfigServiceDNSStore.java not according to RFC4034

[GoogleCodeExporter]

What steps will reproduce the problem?
1. create a pkcs12 file with a RSA private key
2. add certificate via config-ui
3. request certification via dig command:
     "dig @<ip-of-host> <health-domain-name> CERT"

What is the expected output? What do you see instead?
Expected to see a cert resource record with RSASHA1 and with key tag calculated 
according to RFC4034
Saw instead: unexpected key tag value

What version of the product are you using? On what operating system?
direct-project-stock-1.2
Ubuntu

Please provide any additional information below.

in ConfigServiceDNSStore.java:
X509Certificate xCert = dataToCert(cert.getData());
  int keyTag = 0;
  if (xCert.getPublicKey() instanceof RSAKey)
    {
       RSAKey key = (RSAKey)xCert.getPublicKey();
       byte[] modulus = key.getModulus().toByteArray();
       keyTag = (modulus[modulus.length - 2] << 8) & 0xFF00;
       keyTag |= modulus[modulus.length - 1] & 0xFF;
    }

Should be, except for RSA/MD5:
translate_from_C_to_Java(

unsigned int
keytag (
    unsigned char key[],  /* the RDATA part of the DNSKEY RR */
    unsigned int keysize  /* the RDLENGTH */
       )
  {
    unsigned long ac;     /* assumed to be 32 bits or larger */
    int i;                /* loop index */

    for ( ac = 0, i = 0; i < keysize; ++i )
           ac += (i & 1) ? key[i] : key[i] << 8;
    ac += (ac >> 16) & 0xFFFF;
    return ac & 0xFFFF;
  }

)

Original issue reported on code.google.com by NFinst...@gmail.com on 16 Aug 2011 at 4:59

[GoogleCodeExporter]

Original comment by gm2...@cerner.com on 8 Sep 2011 at 7:20

• Changed state: Accepted

[GoogleCodeExporter]

Was this ever resolved?

Original comment by BrianCHo...@gmail.com on 6 Jun 2013 at 3:07