Saved pdf by pdf-lib and after apply pdf-sign with certificate p12 return corrupted pdf

[brasycad]

If I use this PdfSign class in any pdf, it generates a perfectly signed pdf with the p12 certificate. But if that pdf has gone through the pdf-lib library, and is simply saved, and then the PdfSign class is applied, it generates a corrupt pdf. The same pdf that was previously certified correctly. Which is different between the original pdf and the saved by pdf-lib so that this is no longer valid to be signed? Thanks very much. Your work at this library is excellent. Congratulations.

`declare var forge, pdfjsCoreDocument, sha256

export class PdfSign { private root private rootSuccessor private pdf private p12 private date: Date signpdfWithP12(pdfRaw: Uint8Array | ArrayBuffer, p12, date = new Date()): Uint8Array { const loadPdf = (pdfArray: Uint8Array) => { const pdf = new pdfjsCoreDocument.PDFDocument(false, pdfArray, ''); pdf.parseStartXRef(); pdf.parse(); return pdf; } this.p12 = p12 this.date = date if (pdfRaw instanceof ArrayBuffer) pdfRaw = new Uint8Array(pdfRaw); this.pdf = loadPdf(pdfRaw as Uint8Array); this.root = this.findRootEntry(this.pdf.xref); this.rootSuccessor = this.findSuccessorEntry(this.pdf.xref.entries, this.root); return this.isSigInRoot(this.pdf) ? this.appendSigWithP12() : this.newSigWithP12() } signWithP12(rawpdf) { const certBag = '1.2.840.113549.1.12.10.1.3'; const keyBag = '1.2.840.113549.1.12.10.1.2'; let bags = this.p12.getBags({ bagType: certBag }); const p7 = forge.pkcs7.createSignedData(); p7.content = forge.util.createBuffer(rawpdf); let last = bags[certBag][0]; for (let i in bags[certBag]) { p7.addCertificate(bags[certBag][i].cert); last = bags[certBag][i]; } bags = this.p12.getBags({ bagType: keyBag }); const bag = bags[keyBag][0]; const key = bag.key; p7.addSigner({ key: key, certificate: last.cert, digestAlgorithm: forge.pki.oids.sha256, authenticatedAttributes: [{ type: forge.pki.oids.contentType, value: forge.pki.oids.data }, { type: forge.pki.oids.messageDigest }, { type: forge.pki.oids.signingTime, value: this.date }] }); p7.signDetached(); const raw = forge.asn1.toDer(p7.toAsn1()).getBytes(); const hex = this.strHex(raw); return hex; } createXrefTable(xrefEntries) { xrefEntries = this.sortOnKeys(xrefEntries); let retVal = 'xref\n'; let last = -2; for (let key in xrefEntries) { let i = parseInt(key) if (typeof xrefEntries[i].offset === 'undefined') continue; retVal += this.calcFlow(i, last, xrefEntries); const offset = xrefEntries[i].offset; retVal += this.pad10(offset) + ' ' + this.pad5(xrefEntries[i].gen) + ' ' + (xrefEntries[i].free ? 'f' : 'n') + ' \n'; last = Number(i); } return retVal; } calcFlow(i, last, xrefEntries) { if (last + 1 === i) return ''; let count = 1; while (typeof xrefEntries[(i + count)] !== 'undefined' && typeof xrefEntries[(i + count)].offset !== 'undefined') { count++; } return i + ' ' + count + '\n'; } createTrailer(topDict, startxref, sha256Hex, size, prev = null) { let retVal = 'trailer <<\n'; retVal += ' /Size ' + (size) + '\n'; const refRoot = topDict.getRaw('Root'); if (typeof refRoot !== 'undefined') retVal += ' /Root ' + refRoot.num + ' ' + refRoot.gen + ' R\n'; var refInfo = topDict.getRaw('Info'); if (typeof refInfo !== 'undefined') retVal += ' /Info ' + refInfo.num + ' ' + refInfo.gen + ' R\n'; retVal += ' /ID [<' + sha256Hex.substring(0, 32) + '><' + sha256Hex.substring(32, 64) + '>]\n'; if (prev) retVal += ' /Prev ' + prev + '\n'; retVal += '>>\n'; retVal += 'startxref\n'; retVal += startxref + '\n'; retVal += '%%EOF\n'; return retVal; } createXrefTableAppend(xrefEntries) { xrefEntries = this.sortOnKeys(xrefEntries); var retVal = 'xref\n'; var last = -2; for (var key in xrefEntries) { let i = parseInt(key); if (typeof xrefEntries[i].offset === 'undefined') continue; retVal += this.calcFlow(i, last, xrefEntries); var offset = xrefEntries[i].offset; retVal += this.pad10(offset) + ' ' + this.pad5(xrefEntries[i].gen) + ' ' + (xrefEntries[i].free ? 'f' : 'n') + ' \n'; last = Number(i); } return retVal; } sortOnKeys(dict) { const sorted = []; for (var key in dict) sorted[sorted.length] = key; sorted.sort(); var tempDict = {}; for (var i = 0; i < sorted.length; i++) tempDict[sorted[i]] = dict[sorted[i]]; return tempDict; } removeFromArray(array, from, to) { let cutlen = to - from; let buf = new Uint8Array(array.length - cutlen); for (let i = 0; i < from; i++) buf[i] = array[i]; for (let i = to, len = array.length; i < len; i++) buf[i - cutlen] = array[i]; return buf; } findXrefBlocks(xrefBlocks) { let num = xrefBlocks.length / 2; let retVal = []; for (var i = 0; i < num; i++) retVal.push({ start: xrefBlocks[i], end: xrefBlocks[i + num] }); return retVal; } convertUint8ArrayToBinaryString(u8Array) { var i, len = u8Array.length, b_str = ""; for (i = 0; i < len; i++) b_str += String.fromCharCode(u8Array[i]); return b_str; } arrayObjectIndexOf(array, start, end, orig) { for (var i = 0, len = array.length; i < len; i++) if ((array[i].start === start) && (array[i].end === end) && (array[i].orig === orig)) return i; return -1; } pad10(num) { const s = ("000000000" + num); return s.substr(s.length - 10); } pad5(num) { const s = "0000" + num; return s.substr(s.length - 5); } pad2(num) { const s = "0" + num; return s.substr(s.length - 2); } findRootEntry(xref) { var rootNr = xref.root.objId.substring(0, xref.root.objId.length - 1); return xref.entries[rootNr]; } findSuccessorEntry(xrefEntries, current) { const currentOffset = current.offset; let currentMin: number = Number.MAX_SAFE_INTEGER; let currentMinIndex: number = -1; for (let i in xrefEntries) { if (xrefEntries[i].offset > currentOffset) { if (xrefEntries[i].offset < currentMin) { currentMin = xrefEntries[i].offset; currentMinIndex = Number(i); } } } return (currentMinIndex === -1) ? current : xrefEntries[currentMinIndex]; } updateArray(array, pos, str) { const upd = this.stringToUint8Array(str); for (let i = 0, len = upd.length; i < len; i++) array[i + pos] = upd[i]; return array; } copyToEnd(array, from, to) { var buf = new Uint8Array(array.length + (to - from)); for (let i = 0, len = array.length; i < len; i++) buf[i] = array[i]; for (let i = 0, len = (to - from); i < len; i++) buf[array.length + i] = array[from + i]; return buf; } insertIntoArray(array, pos, str) { const ins = this.stringToUint8Array(str); const buf = new Uint8Array(array.length + ins.length); for (let i = 0; i < pos; i++) buf[i] = array[i]; for (let i = 0; i < ins.length; i++) buf[pos + i] = ins[i]; for (let i = pos; i < array.length; i++) buf[ins.length + i] = array[i]; return buf; } stringToUint8Array(str) { const buf = new Uint8Array(str.length); for (let i = 0, strLen = str.length; i < strLen; i++) buf[i] = str.charCodeAt(i); return buf; } uint8ArrayToString(buf, from, to) { if (typeof from !== 'undefined' && typeof to !== 'undefined') { let s = ''; for (let i = from; i < to; i++) s = s + String.fromCharCode(buf[i]); return s; } return String.fromCharCode.apply(null, buf); } findFreeXrefNr(xrefEntries, used = []) { var inc = used.length; for (var i = 1; i < xrefEntries.length; i++) { var index = used.indexOf(i); var entry = xrefEntries["" + i]; if (index === -1 && (typeof entry === 'undefined' || entry.free)) return i; if (index !== -1) inc--; } return xrefEntries.length + inc; } find(uint8, needle, start = 0, limit = Number.MAX_SAFE_INTEGER) { const search = this.stringToUint8Array(needle); let match = 0; for (var i = start; i < uint8.length && i < limit; i++) { if (uint8[i] === search[match]) { match++; } else { match = 0; if (uint8[i] === search[match]) { match++; } } if (match === search.length) return (i + 1) - match; } return -1; } findBackwards(uint8, needle, start = uint8.length, limit = Number.MAX_SAFE_INTEGER) { const search = this.stringToUint8Array(needle); let match = search.length - 1; for (var i = start; i >= 0 && i < limit; i--) { if (uint8[i] === search[match]) { match--; } else { match = search.length - 1; if (uint8[i] === search[match]) { match--; } } if (match === 0) return i - 1; } return -1; } strHex(s) { var a = ""; for (var i = 0; i < s.length; i++) a = a + this.pad2(s.charCodeAt(i).toString(16)); return a; } isSigInRoot(pdf) { if (typeof pdf.acroForm === 'undefined') return false; return pdf.acroForm.get('SigFlags') === 3; } updateXrefOffset(xref, offset, offsetDelta) { for (var i in xref.entries) if (xref.entries[i].offset >= offset) xref.entries[i].offset += offsetDelta; for (var i in xref.xrefBlocks) if (xref.xrefBlocks[i] >= offset) xref.xrefBlocks[i] += offsetDelta; } updateXrefBlocks(xrefBlocks, offset, offsetDelta) { for (var i in xrefBlocks) { if (xrefBlocks[i].start >= offset) xrefBlocks[i].start += offsetDelta; if (xrefBlocks[i].end >= offset) xrefBlocks[i].end += offsetDelta; } } updateOffset(pos, offset, offsetDelta) { return (pos >= offset) ? pos + offsetDelta : pos; } round256(x) { return (Math.ceil(x / 256) 256) - 1; } now(date = new Date()) { const yyyy = date.getFullYear().toString(); const MM = this.pad2(date.getMonth() + 1); const dd = this.pad2(date.getDate()); const hh = this.pad2(date.getHours()); const mm = this.pad2(date.getMinutes()); const ss = this.pad2(date.getSeconds()); return yyyy + MM + dd + hh + mm + ss + this.createOffset(date); } createOffset(date) { const sign = (date.getTimezoneOffset() > 0) ? "-" : "+"; const offset = Math.abs(date.getTimezoneOffset()); const hours = this.pad2(Math.floor(offset / 60)); const minutes = this.pad2(offset % 60); return sign + hours + "'" + minutes; } newSigWithP12(): Uint8Array { const annotEntry = this.findFreeXrefNr(this.pdf.xref.entries); const offsetForm = this.find(this.pdf.stream.bytes, '<<', this.root.offset, this.rootSuccessor.offset) + 2; const appendAcroForm = '/AcroForm<</Fields[' + annotEntry + ' 0 R] /SigFlags 3>>'; const pages = this.pdf.catalog.catDict.get('Pages'); const ref = pages.get('Kids')[0]; const xref = this.pdf.xref.fetch(ref); const offsetContentEnd = xref.get('#Contents_offset'); let offsetContent = this.findBackwards(this.pdf.stream.bytes, '/Contents', offsetContentEnd); const appendAnnots = '/Annots[' + annotEntry + ' 0 R]\n '; let array = this.insertIntoArray(this.pdf.stream.bytes, offsetForm, appendAcroForm); this.updateXrefOffset(this.pdf.xref, offsetForm, appendAcroForm.length); offsetContent = this.updateOffset(offsetContent, offsetForm, appendAcroForm.length); array = this.insertIntoArray(array, offsetContent, appendAnnots); this.updateXrefOffset(this.pdf.xref, offsetContent, appendAnnots.length); offsetContent = -1; //not needed anymore, don't update when offset changes const sigEntry = this.findFreeXrefNr(this.pdf.xref.entries, [annotEntry]); const append = annotEntry + ' 0 obj\n<</F 132/Type/Annot/Subtype/Widget/Rect[0 0 0 0]/FT/Sig/DR<<>>/T(signature' + annotEntry + ')/V ' + sigEntry + ' 0 R>>\nendobj\n\n'; const blocks = this.findXrefBlocks(this.pdf.xref.xrefBlocks); let offsetAnnot = blocks[0].start; array = this.insertIntoArray(array, offsetAnnot, append); let offsetSig = offsetAnnot + append.length; const start = sigEntry + ' 0 obj\n<</Contents <'; const dummy = this.signWithP12('A'); const crypto = new Array(this.round256(dummy.length 2)).join('0'); const middle = '>\n/Type/Sig/SubFilter/adbe.pkcs7.detached/Location()/M(D:' + this.now(this.date) + '\')\n/ByteRange '; let byteRange = '[0000000000 0000000000 0000000000 0000000000]'; const end = '/Filter/Adobe.PPKLite/Reason()/ContactInfo()>>\nendobj\n\n'; const append2 = start + crypto + middle + byteRange + end; const offsetByteRange = start.length + crypto.length + middle.length; array = this.insertIntoArray(array, offsetSig, append2); this.updateXrefOffset(this.pdf.xref, offsetAnnot, append2.length + append.length); const xrefBlocks = this.findXrefBlocks(this.pdf.xref.xrefBlocks); for (let i in xrefBlocks) { const oldSize = array.length; array = this.removeFromArray(array, xrefBlocks[i].start, xrefBlocks[i].end); const length = array.length - oldSize; this.updateXrefOffset(this.pdf.xref, xrefBlocks[i].start, length); const offsetEOF = this.find(array, '%%EOF', xrefBlocks[i].start, xrefBlocks[i].start + 20); if (offsetEOF > 0) { var lengthEOF = '%%EOF'.length; array = this.removeFromArray(array, offsetEOF, offsetEOF + lengthEOF); this.updateXrefOffset(this.pdf.xref, offsetEOF, -lengthEOF); this.updateXrefBlocks(xrefBlocks, offsetEOF, -lengthEOF); offsetAnnot = this.updateOffset(offsetAnnot, offsetEOF, -lengthEOF); offsetSig = this.updateOffset(offsetSig, offsetEOF, -lengthEOF); } this.updateXrefBlocks(xrefBlocks, xrefBlocks[i].start, length); offsetAnnot = this.updateOffset(offsetAnnot, xrefBlocks[i].start, length); offsetSig = this.updateOffset(offsetSig, xrefBlocks[i].start, length); } const sha256Hex = sha256(array, false); this.pdf.xref.entries[annotEntry] = { offset: offsetAnnot, gen: 0, free: false }; this.pdf.xref.entries[sigEntry] = { offset: offsetSig, gen: 0, free: false }; let xrefTable = this.createXrefTable(this.pdf.xref.entries); xrefTable += this.createTrailer(this.pdf.xref.topDict, array.length, sha256Hex, this.pdf.xref.entries.length); array = this.insertIntoArray(array, array.length, xrefTable); let from1 = 0; const to1 = offsetSig + start.length; const from2 = to1 + crypto.length; const to2 = (array.length - from2) - 1; byteRange = '[' + this.pad10(from1) + ' ' + this.pad10(to1 - 1) + ' ' + this.pad10(from2 + 1) + ' ' + this.pad10(to2) + ']'; array = this.updateArray(array, (offsetSig + offsetByteRange), byteRange); const data = this.removeFromArray(array, to1 - 1, from2 + 1); const crypto2 = this.signWithP12(data); return this.updateArray(array, to1, crypto2); } appendSigWithP12() { var startRoot = this.pdf.stream.bytes.length + 1; var array = this.copyToEnd(this.pdf.stream.bytes, this.root.offset - 1, this.rootSuccessor.offset); var offsetAcroForm = this.find(array, '/AcroForm<</Fields', startRoot); var endOffsetAcroForm = this.find(array, ']', offsetAcroForm); var annotEntry = this.findFreeXrefNr(this.pdf.xref.entries); var sigEntry = this.findFreeXrefNr(this.pdf.xref.entries, [annotEntry]); var appendAnnot = ' ' + annotEntry + ' 0 R'; array = this.insertIntoArray(array, endOffsetAcroForm, appendAnnot); var pages = this.pdf.catalog.catDict.get('Pages'); var contentRef = pages.get('Kids')[0]; var xref = this.pdf.xref.fetch(contentRef); var offsetAnnotEnd = xref.get('#Annots_offset'); var endOffsetAnnot = this.find(array, ']', offsetAnnotEnd); var xrefEntry = this.pdf.xref.getEntry(contentRef.num); var xrefEntrySuccosser = this.findSuccessorEntry(this.pdf.xref.entries, xrefEntry); var offsetAnnotRelative = endOffsetAnnot - xrefEntrySuccosser.offset; var startContent = array.length; array = this.copyToEnd(array, xrefEntry.offset, xrefEntrySuccosser.offset); array = this.insertIntoArray(array, array.length + offsetAnnotRelative, appendAnnot); var startAnnot = array.length; var append = annotEntry + ' 0 obj\n<</F 132/Type/Annot/Subtype/Widget/Rect[0 0 0 0]/FT/Sig/DR<<>>/T(signature' + annotEntry + ')/V ' + sigEntry + ' 0 R>>\nendobj\n\n'; array = this.insertIntoArray(array, startAnnot, append); var startSig = array.length; var start = sigEntry + ' 0 obj\n<</Contents <'; var dummy = this.signWithP12('A'); var crypto = new Array(this.round256(dummy.length * 2)).join('0'); var middle = '>\n/Type/Sig/SubFilter/adbe.pkcs7.detached/Location()/M(D:' + this.now(this.date) + '\')\n/ByteRange '; var byteRange = '[0000000000 0000000000 0000000000 0000000000]'; var end = '/Filter/Adobe.PPKLite/Reason()/ContactInfo()>>\nendobj\n\n'; var append2 = start + crypto + middle + byteRange + end; array = this.insertIntoArray(array, startSig, append2); const sha256Hex = sha256(array, false); var prev = this.pdf.xref.xrefBlocks[0]; var startxref = array.length; var xrefEntries = []; xrefEntries[0] = { offset: 0, gen: 65535, free: true }; xrefEntries[this.pdf.xref.topDict.getRaw('Root').num] = { offset: startRoot, gen: 0, free: false }; xrefEntries[contentRef.num] = { offset: startContent, gen: 0, free: false }; xrefEntries[annotEntry] = { offset: startAnnot, gen: 0, free: false }; xrefEntries[sigEntry] = { offset: startSig, gen: 0, free: false }; var xrefTable = this.createXrefTableAppend(xrefEntries); xrefTable += this.createTrailer(this.pdf.xref.topDict, startxref, sha256Hex, xrefEntries.length, prev); array = this.insertIntoArray(array, array.length, xrefTable); var from1 = 0; var to1 = startSig + start.length; var from2 = to1 + crypto.length; var to2 = (array.length - from2) - 1; var byteRange = '[' + this.pad10(from1) + ' ' + this.pad10(to1 - 1) + ' ' + this.pad10(from2 + 1) + ' ' + this.pad10(to2) + ']'; array = this.updateArray(array, from2 + middle.length, byteRange); var data = this.removeFromArray(array, to1 - 1, from2 + 1); var crypto2 = this.signWithP12(data); return this.updateArray(array, to1, crypto2); } }`

[therpobinski]

I have the same problem, I think I found the cause, pdf-lib removes the trailer from the pdf and when signature does not find the trailer to add the signature. Do you have any solution?

I am working with the node-signpdf library and I have just made a pull-request where I use pdf-lib and there is an error when I want to sign. Here I explain a little about this pull-request.

[brasycad]

Yes, I have also detected that there is no trailer in the new document saved by pdf-lib. I don't know why this happens. But I'm going to go deeper into the PDF-lib code!!

[therpobinski]

I have advanced with this issue, from another project, I first try to put a placeholder with a widget and sign the document, although I still have 3 errors so I have not been able to move forward, so you can help me. Here is more information than I say. plainAddPlaceholder#comment

[vbuch]

If you read this issue, you will know why there is no trailer.

[vbuch]

@brasycad did you succeed? Because the lib still does not support object streams. Interested in why you closed it.

[zerobytes]

it is April, already, but i'm facing the same problem. Signing the PDF is failing because the trailer is missing.

[Hopding]

This issue relates to https://github.com/Hopding/pdf-lib/issues/278#issuecomment-571387464

[DvirH]

Did you find any solution for this issue? I face with the same problem with no solution. I tried to manipulate the byte array by adding the trailer myself but node-signpdf still throw an exception.

[DvirH]

This issue relates to #278 (comment)

I Don't think it's about updating a signed file. In order to sign a file using the node-signpdf library, the library is looking for the trailer characters, and after loading an existing pdf and save it using pdf-lib it seems the library removes the trailer part. Do you have and idea why is that?

[therpobinski]

before manipulating the pdf, did you see if it has a trailer? I have noticed that in versions higher than 1.3 in the PDF, it no longer works with the cross references and the trailer. Well, to read a PDF you don't need these parameters, but to sign them you do. I have not found a way to sign PDF with a version higher than 1.3, or in itself, PDF that does not comply with the structure mentioned here

[aallvi]

anybody can find a solution? or we are waiting for an update? how you have a solution? use another library? etc, please help, i need to sign a a pdf previusly signed