henrinormak
commented
7 years ago The padding is chosen here (line 211) for encryption and here (line 297) for decryption. You should be able to change them, but only if you change both of them. You should be able to run the tests to verify that this will indeed produce expected results (which it should, as Heimdall at some point DID use PCKS1).
The two values MUST be the same, as otherwise values encrypted will not be properly decrypted. Also note, that the padding calculation for PKCS1 should be different (lines 300 and 214) should use the value 11 instead of 42 AFAIK.
I'm attempting to integrate a fork of Heimdall with some other platforms that don't have the ability to do OAEP (long story short, the Android keystore didn't support it until a recent SDK version). I see a comment in the code that calculates the AES key size that says "Assumes SHA1-OAEP is used", but it's unclear to me what logic or values are based on that assumption. Say I were to switch to using
SecPadding.PKCS1, would something in this logic need to be updated? Thanks in advance for any info you can provide.