WIP: RAM usage reduction

[promovicz]

WORK IN PROGRESS! DO NOT MERGE YET!

The OpenPGP applet currently uses two large buffers, "tmp" and "buffer". The total RAM usage of the applet makes it impossible to install and use it on most common JavaCard models. More specifically only a few JCOP cards have enough memory to run the applet.

After some research I found that "tmp" can be eliminated entirely. This patch series is a work in progress to eliminate it entirely, which in itself is working on my branch.

The "tmp" buffer is used in four places:

• As a serialization buffer in sendPublicKey. This usage can be eliminated by using a better serialization algorithm that predicts memory size before starting to serialize.
• As an operational buffer in computeDigitalSignature, decipher and internalAuthenticate. This usage can be eliminated by recognizing that "buffer" is always large enough since the actual message size is always 256 for RSA-based operations with 2048bit keys. The current buffer would also be large enough for 4096bit operations, should they become supported.

My plan is to add additional size checks for added safety.

[promovicz]

I would of course love your comments on this patch and an indication on whether you would merge it once completed.

[klali]

Hello,

Without looking to deeply I'd say that yes, we'd welcome a change like this. Let us know when you believe it's ready to be reviewed and merged.

/klas

[coveralls]

Coverage increased (+0.36%) to 45.09% when pulling 06453080b7e60294c1af69b9183ef1a3f3e54830 on promovicz:ram-usage-reduction into daa0c1094d547a8e846dd24180739d1df8853299 on Yubico:master.

[coveralls]

Coverage increased (+0.62%) to 45.36% when pulling 9c98f737bd9e5eb83813154af255d2f27cad65b3 on promovicz:ram-usage-reduction into daa0c1094d547a8e846dd24180739d1df8853299 on Yubico:master.

[jonathancross]

@promovicz Is this abandoned?

[dogtopus]

Apparently someone else gave it a shot (8f7353dc)