Possible performance issue with BER encoding

[rideckard]

Hi,

Using "cms Enveloped Data" the BER encoding is really slow, I am measuring ~15 sek for encrypted 2mb buffer in this step: cmsEnvelopedBuffer = cmsContentSimpl.toSchema().toBER(false);

the data Part in the asn.1 schema looks like this

valueBlock: Object { blockName: "OctetStringValueBlock", blockLength: 0, isIndefiniteForm: false, … }​​​​​​​​​​​
blockLength: 0
​​​​​​​​​​​​​​blockName: "OctetStringValueBlock"
​​​​​​​​​​​​​​error: ""
​​​​​​​​​​​​​​isConstructed: true
​​​​​​​​​​​​​​isHexOnly: false
​​​​​​​​​​​​​​isIndefiniteForm: false
​​​​​​​​​​​​​​value: Array(1344) [ {…}, {…}, {…}, … ]
​​​​​​​​​​​​​​​[0…99]
​​​​​​​​​​​​​​​​    0: Object { blockName: "OCTET STRING", blockLength: 0, error: "", … }
​​​​​​​​​​​​​​​​​        blockLength: 0
​​​​​​​​​​​​​​​        blockName: "OCTET STRING"
​​​​​​​​​​​​​​​​​        error: ""
​​​​​​​​​​​​​​​​​        idBlock: Object { blockName: "identificationBlock", blockLength: 0, isHexOnly: false, … }
​​​​​​​​​​​​​​​​​        lenBlock: Object { blockName: "lengthBlock", blockLength: 0, isIndefiniteForm: false, … }
​​​​​​​​​​​​​​        valueBeforeDecode: ""
​​​​​​​​​​​​​​​​        valueBlock: Object { blockName: "OctetStringValueBlock", blockLength: 0, isIndefiniteForm: false, … }
​​​​​​​​​​              blockLength: 0
​              blockName: "OctetStringValueBlock"
​​​​​​​​​​​​​​​​​​              error: ""
​​​​​​​​​​​​​​              isConstructed: false
​​​​​​​​              isHexOnly: false
​​​​​​​​​​​​​​​​              isIndefiniteForm: false
​​​​​              value: Array []
​​​​​​​​​​​​              valueBeforeDecode: ""
​​​​​​​​​​​​​              valueHex:  "E30CE9F60944EA0E9F324BD71E16307FD5DBE745780FCB124D0523B9940E8809...."
​​​​​​​​​​​​​​​​​              warnings: Array []
[100...199] ...
...

As far as I understand the BER encoding with asn1js (which is not very far:) it should only concatenate those 1344 byte strings. Why would it take so much time? Did I do something something wrong? Sorry if this is the wrong place for this question, but I am quite desperate to figure this out and apart from the slow encoding everything works perfectly!

[microshine]

Looks like the problem is in asn1js

JS example (2Mb)

const dataToEncrypt = new Uint8Array(2 * 1024 * 1024).buffer;

const cmsEnveloped = new pkijs.EnvelopedData();

// Add recipient
cmsEnveloped.addRecipientByCertificate(cert, { oaepHashAlgorithm: "SHA-256" });

// Secret key algorithm
const alg = {
  name: "AES-GCM",
  length: 256,
}
console.time("cmsEnveloped.encrypt");
await cmsEnveloped.encrypt(alg, dataToEncrypt);
console.timeEnd("cmsEnveloped.encrypt");

// Add Enveloped Data into CMS Content Info
console.time("cmsEnveloped.toSchema()");
const cmsContent = new pkijs.ContentInfo();
cmsContent.contentType = pkijs.ContentInfo.ENVELOPED_DATA;
cmsContent.content = cmsEnveloped.toSchema();
console.timeEnd("cmsEnveloped.toSchema()");

console.time("cmsContent.toSchema().toBER()");
const cmsContentRaw = cmsContent.toSchema().toBER();
console.timeEnd("cmsContent.toSchema().toBER()");

Console output

cmsEnveloped.encrypt: 31.459ms
cmsEnveloped.toSchema(): 0.619ms
cmsContent.toSchema().toBER(): 6.416s

[microshine]

Here is the result of some researches.

It's not optimized implementation of toBER method in asn1js

https://github.com/PeculiarVentures/ASN1.js/blob/master/src/asn1.js#L1258

    toBER(sizeOnly = false)
    {
        let retBuf = new ArrayBuffer(0);

        for(let i = 0; i < this.value.length; i++)
        {
            const valueBuf = this.value[i].toBER(sizeOnly); // this line should be optimized
            retBuf = utilConcatBuf(retBuf, valueBuf);
        }

        return retBuf;
    }

It rewrites retBuf value N time (where N is this.value.length). App should do it only once

I applied that change to my local instance of asn1js

toBER(sizeOnly = false) {
    let retBufs = [];
    for (let i = 0; i < this.value.length; i++) {
      const valueBuf = this.value[i].toBER(sizeOnly);
      retBufs.push(valueBuf);
    }

    return _pvutils.utilConcatBuf(...retBufs);
  }

Console output (2Mb)

cmsEnveloped.encrypt: 29.454ms
cmsEnveloped.toSchema(): 0.494ms
cmsContent.toSchema().toBER(): 130.639ms

Console output (10Mb)

cmsEnveloped.encrypt: 99.636ms
cmsEnveloped.toSchema(): 0.624ms
cmsContent.toSchema().toBER(): 558.819ms

[microshine]

@rideckard I've published the new version of asn1js@2.3.3. Please update your dependencies and try it. It must work faster

[microshine]

@rideckard I've published one more new version of asn1js@2.4.0

Now it encodes ASN.1 structures faster

Console output (10Mb)

cmsEnveloped.encrypt: 84.205ms
cmsEnveloped.toSchema(): 0.537ms
cmsContent.toSchema().toBER(): 75.482ms

[rideckard]

Brilliant! Great work, its lightning fast now. I can confirm your benchmark results. You just made my day!