Signatures: Difficulty verifying B.2 examples

[bblfish]

I have implemented nearly all of "Signing HTTP Messages v04" and have tested it carefully in TestMessageSigningRFCFn.scala. All the tests I wrote pass, except for the signature verification of the examples in Appendix B.

The 3 problematic cases are commented out here:

• line 392

  • line 450
  • line 514

  In each case I am able to sign and verify the headers myself using the Java Cryptography API, using the rsa-pss-sha512 Signature object. It may be that I am not using the right signature algorithm, or it may be that the signatures in the spec are not quite right, or ... Since the spec is still in development I thought I would check here.

  This is the signature I tried out:

  lazy val `rsa-pss-sha512`: JSignature =
  //is this the same as JWT PS512?
  //see [[https://tools.ietf.org/html/rfc7518 JSON Web Algorithms (JWA) RFC]]
  val rsapss = JSignature.getInstance("RSASSA-PSS")
  import java.security.spec.{PSSParameterSpec, MGF1ParameterSpec}
  rsapss.setParameter(new PSSParameterSpec(
      "SHA-512", "MGF1", MGF1ParameterSpec.SHA512, 512 / 8, 1))
  rsapss

Where JSignature is a just an alias for java.security.Signature.

I also have some comments in the test suite regarding other parts of the spec:

• List Prefixes tests: it occurred to me that perhaps Inner Lists were not the right data structure for lists, but that normal RFC8941 lists would be better suited.
  • Multiple Signatures: the example is very interesting. It would be nice to have the full HTTP Message on which it is built. Also there clearly is a mistake in the string written up in the spec.
  • The signature input string example could do with a bit of explanation to clarify if the extra space is needed, as it is not visible in the spec and so could easily be overseen.

[jricher]

All the signatures in the spec in -04 were generated and verified using a python implementation (and we're figuring out the best place to release it to), so they should all work as is -- unless I've got a bug in my own implementation! I'm not deeply familiar with Scala, but it looks like you might be checking signature validity by re-generating the signature and then evaluating for equality with the presented signature, is that correct? If so, then that is not going to work for the RSA PSS algorithm (or most elliptic curve algorithms, which we haven't added examples for). The output of the signing algorithm is not the same for each signing, but the definitions of the verification functions take this into account -- you pass in the signature to be verified, the key, and the content to sign over in order to verify. This is what it looks like in part of the python code that generated the examples:

# Sign
signed = http_sfv.Item(key.sign_rsassa_pss(str.encode(base), algo='sha512'))

# Verify
verified = key.verify_rsassa_pss(str.encode(base), signed.value, algo='sha512')

Here, key is our public/private keypair (we could use just the public key to verify, though), base is the signature base string calculated from the "minimal" example in B.2.1.

As for the other points:

• yes the list prefix examples should be over a top level list not an inner list, thanks for catching that. Can you please file a separate issue for this?
• the multiple signatures message is given in the example: it uses an incoming signature field but doesn't sign other parts of the request
• yes the space is required, as per the serialization algorithm at https://www.ietf.org/archive/id/draft-ietf-httpbis-message-signatures-04.html#section-2.5-2.2.2.3; you can think of this as 'connect the identifier and content with ": " and then connect the lines with "\n" to get the output' if you want to think in terms of join() functions or the like. In the example generation code we do:

base = '';
for c in coveredContent:
    i = http_sfv.Item()
    i.parse(c.encode())
    sigparams.append(i)
    base += c # already serialized as an Item
    base += ': '
    base += coveredContent[c]
    base += "\n"

This is followed by adding the @signature-params item as the last on the list, so we don't have to account for the final line.

[bblfish]

@jricher wrote

All the signatures in the spec in -04 were generated and verified using a python implementation (and we're figuring out the best place to release it to), so they should all work as is -- unless I've got a bug in my own implementation!

It is very helpful to know that these signatures are verified: that gave me some incentive to look carefully at my code.

The tests I link to above are testing too much of the abstraction, making it difficult to duplicate the problem. So I put together a barebones test, to test just the signatures (see commit).

Since that still relies on my environement, I put that together in one simple independent Gist that one can run in a Scala console. This really relies mostly on Java only libraries, and I could give a pure Java version if needed too. This could be something that a Java Crypto person should be able to look at easily. The last line returns false for me, i.e. the signature does not verify.

Am I doing something wrong with the way I am setting up the signature algorithm?

[jricher]

@bblfish Thank you for supplying the gist! In my explorations I found two things:

1. The M2Crypto library I used uses a default salt length of 20; this should be specified (at whatever an appropriate value is) in the spec text's algorithm definition.
2. The M2Crypto library apparently expects you to do the hash ahead of passing it in to the signature function, and the Java version (that you used) does requires the hash ahead of verification. I tried doing the hash first on both the python and Scala sides and got both of them to verify the signature generated by Python, so that's progress. I'm not sure if the spec text needs to be changed but all the examples should certainly be updated.

I need to do some research on the python library first to make sure I'm actually using it correctly now, but I think this is putting us on the right path!

[bblfish]

Can you share the snippet of code that worked in Java?

[jricher]

https://gist.github.com/jricher/f2b330f94e5732663fc1e7bef0c78760

[bblfish]

Btw, as this spec is very likely going to be used a lot with JWK I wonder if it may not be better just to settle on the algorithm names used there, and in fact also publish the keys in those formats, as if they had been published on the web that way, or placed in some DPOP token... I find that ideally keys should not be hidden in so many layers of binary encoding as one has in the Mime encoding. It makes what is already quite complicated even more mysterious... Perhaps if there are two names for the same thing one should mention that. Just to help developers. :-)

[jricher]

@bblfish we've discussed re-using the JWS algorithm identifiers (like RS256) but ultimately didn't want to be bound to that use case. There's a lot of the web that doesn't use JOSE or JWK and HTTP Sig needs to be compatible with that world also and be able to incorporate different algorithm schemes.

We settled on defining a way to re-use the algorithms without leaning on the same registry, in https://www.ietf.org/archive/id/draft-ietf-httpbis-message-signatures-04.html#name-json-web-signature-jws-algo ... we'd expect other algorithm listings like PASETO versions or COSE algorithms or whatever else that makes sense to be able to be folded in here. We also need to account for the use case where expected algorithm is simply fixed by both parties and not negotiated as part of the protocol. That's why the algorithm field is now optional instead of required.

The other algorithms that are explicitly listed are not intended to be just copies of JWA algorithm entries, and this list could be extended independently from JWA. We don't anticipate registering all of JWA either, especially since if you're using it with a JWK, as you note, you can just get the algorithm field from the key itself and use that. So ultimately if you're in the JOSE world, it should make sense and feel JOSE-native, more or less. If you're outside of JOSE, it shouldn't feel like you need to learn JOSE to make it work.

[jricher]

^-- all that said, I'm in favor of having more examples, including a JOSE-native one where the key is in JWK format and the algorithm is specified internally to the key. Developers tend to look at the examples first and then the actual spec later, in my experience. :)

[jricher]

Additional signature parameters have been defined with feedback from CFRG, updated in #1528