The 'undefined' option for pdfToCairo's second parameter does not produce valid output for tiff files.

wunderkind2k1 commented 2 years ago

Prerequisites

[X] I have written a descriptive issue title
[X] I have searched existing issues to ensure it has not already been reported
[X] I agree to follow the Code of Conduct that this project adheres to

API/app/plugin version

5.1.6

Node.js version

v16.13.2

Operating system

macOS

Operating system version (i.e. 20.04, 11.3, 10)

12.4

Description

I am trying to fetch a single page from a pdf without writing it down in a separate image file. This works beautifully for jpg's and png's as documented in https://github.com/Fdawgs/node-poppler/blob/master/README.md#popplerpdftocairo. For tiff files it's a different story though: It works only if I give an output file as a second parameter in the pdfToCairo function, but not when I use 'undefined'. The result ist way to small - like it is only the header of the tiff file or something.

I checked wether my poppler version (22.05.0) is working correctly on the comand line. It does. pdftocairo -tiff -f 1 -l 1 -singlefile example.pdf - > example.tiff works perfectly on the shell.

As far as I can see - node-poppler does send the correct params to the spawned child process. But the result is a very short string - sth like this:

From debugging index.js in node-poppler I can see that

is only called once for the whole childprocess. This could be the problem.

Steps to Reproduce

This code should be enough to see that foo.tif is not a valid tiff file. You can use any or the provided pdf file: example.pdf

import { Poppler } from 'node-poppler';
import fs from 'fs';
import path from 'path';

const file = fs.readFileSync(path.join(__dirname, 'example.pdf'));

(async () => {
  const poppler = new Poppler('/usr/bin');
  const res: string | Error = await poppler.pdfToCairo(file, undefined, {
    firstPageToConvert:1,
    lastPageToConvert: 1,
    singleFile:true,
    tiffCompression: 'jpeg',
    tiffFile: true
    //pngFile: true

  });
  if (res instanceof Error) {
    console.log('Error: ' + JSON.stringify(res));
    return;
  }
  fs.writeFileSync('foo.tif', res, { encoding: 'binary' })
})();

Additional information: Though I wrote this code on OSX I also tried it on a docker container with alpine linux expecting the behaviour to be an OSX glitch. But I could also reproduce the problem on linux successfully.

Expected Behaviour

The expected behaviour should be equal for all possible output formats - meaning when using an 'undefined' outputfile and the -singleFile Option the resulting string should contain valid image data.

Fdawgs commented 2 years ago

Thanks for reporting this @wunderkind2k1, I'll take a look!

wunderkind2k1 commented 2 years ago

Let me know if I can be of any help. Sven

Fdawgs commented 2 years ago

Thanks Sven, I plan on looking at this on Monday. You're more than welcome to have a crack at fixing it yourself and opening a PR if you wish!

Fdawgs commented 2 years ago

Can confirm this is also an issue using Windows 10 with the included binaries:

pdf_1.3_NHS_Constitution.tif is the result when passing a filename to write to, and can be opened
foo.tif is the result when writing to stdout and then writing to file using fs.writeFileSync(), and Windows Photo Viewer throws an error when opened

As you can see, both files are the same size, so no issue with them being cut short. Not really sure on how to fix this. 😞

wunderkind2k1 commented 2 years ago

Ok. Thank you for verifying it. I think its kind of complicated to track down... maybe related to the stream events of the child process? I will try to find some time and have a deeper look into it.

Mustafa-Aswadi commented 2 years ago

hello folks ! I have a question about the path of poppler-utils. how can I add the path of poppler-utils to Poppler object when the application is running in server ? and also can you please provide me the commands of installing poppler data and utils in Alpine os ? thanks. @Fdawgs

wunderkind2k1 commented 2 years ago

hello folks ! I have a question about the path of poppler-utils. how can I add the path of poppler-utils to Poppler object when the application is running in server ? and also can you please provide me the commands of installing poppler data and utils in Alpine os ? thanks. @Fdawgs

Hi Mustafa,

would you please be so kind open and open an other issue for such a question the next time? It does not belong to this issue.

Nevertheless:

"how can I add the path of poppler-utils to Poppler object when the application is running in server ?" -> you put it in the constructor of the Poppler 'object'. like so:

const poppler = new Poppler(<INSERT YOUR PATH TO POPPLER_UTILS>);

"and also can you please provide me the commands of installing poppler data and utils in Alpine os ?" -> this should do the trick:

apk add poppler-utils

Hope this helps.

Mustafa-Aswadi commented 2 years ago

Hi @wunderkind2k1 sorry for using this issue for non-relevant question.

anyway, Thanks for your helps.

wunderkind2k1 commented 1 year ago

Hi. A little update on this: I had the chance to dig deeper into this during the holidays.

I changed src/index.js a little to get an error:

After running this testcode:

const fs = require("fs");
const path = require("path");
const { Poppler } = require("./index");
const findCairoBinaryPath = require("./findCairoBinaryPath");

const file = fs.readFileSync(
    path.join(__dirname, "../test_files/issue_430.pdf")
);

(async () => {
    try {
        const currentPathToPOPPLER = findCairoBinaryPath();
        const poppler = new Poppler(currentPathToPOPPLER);
        const res = await poppler.pdfToCairo(file, undefined, {
            tiffFile: true,
            tiffCompression: "jpeg",
            //pngFile: true,
            singleFile: true,
            firstPageToConvert: 1,
            lastPageToConvert: 1,
        });
        if (res instanceof Error) {
            console.log(`Error:\n${JSON.stringify(res)}`);
            return;
        }
        console.log(res.length);
        fs.writeFileSync("foo.tif", res, { encoding: "binary" });
    } catch (e) {
        console.log(`Error: ${e.toString()}`);
    }
})();

This is the resulting error:

Error: Error: TIFFAppendToStrip: Maximum TIFF file size exceeded.
TiffWriter: Error writing tiff row 16

// repetitions removed

TIFFAppendToStrip: Maximum TIFF file size exceeded.
TiffWriter: Error writing tiff row 1648
JPEGLib: Application transferred too few scanlines.

To me this looks like there is a problem with the underlying libs like libtiff then. But I haven't been able to find the reason yet, especially as I am wondering why a call to

cat test_files/issue_430.pdf | pdftocairo -tiff -tiffcompression jpeg -singlefile -f 1 -l 1 - - > bar.tif

produces a nice bar.tif file then.

Fdawgs commented 1 year ago

Thanks for continuing to look at this @wunderkind2k1!

msageryd commented 1 year ago

@wunderkind2k1 Did you find a solution to this?

I have just added streaming support to poppler.pdfToCairo in a fork. I tried rendering to TIFF and got the same problem. Just wanted to chime in and inform that this does not seem to be a buffer problem, since my stream version behaves the same.

The file size of my TIFF files are 8 bytes. Rendering to JPG files is no problem with file sizes about 3-9 MB

I can confirm that the exact same pdftocario call works fine manually. The resulting TIFF file is 58 MB in my case. I.e. the last argument to my manually called pdftocairo is "-", which will direct the output to stdout. Piping this into a file works great.

The only difference between my manual call and poppler.pdfToCairo is the spawn command.

stdout ends after 8 bytes when -tiff is used
I tried to attach a 'finish' event and I got it. This means that stream.end is called from somewhere.
stdout works great when -jpeg is used

This post might relate to this: https://github.com/nodejs/node/issues/12921 But I didn't find anything useful.

My fork: https://github.com/msageryd/node-poppler

wunderkind2k1 commented 1 year ago

I didn't find a solution to this until now. I ended up doing the tiff conversion with libvips... which has been a solution but a unnecessarily complex one.

msageryd commented 1 year ago

Are you using libvips for converting from pdf->tiff? Or are you using pdftocario for png export and libvips for png->tiff?

If the latter is true, you might want to use my stream version, since this makes it easy to pipe the result from poppler.pdfToCairo straigth into Sharp (Sharp uses libvips).

wunderkind2k1 commented 1 year ago

I extracted a png with poppler and used sharp to convert it to a tiff which than has been fed into an ocr... But I am not working on that project anymore. Thanks for sharing your lib - I might use it if needed <3

mvz commented 8 months ago

This issue may be related: https://gitlab.freedesktop.org/poppler/poppler/-/issues/1334.

TL;DR: The tiff library needs random access to the output file.

wunderkind2k1 commented 8 months ago

That sounds like the explanation! Thank you for sharing!

Fdawgs / node-poppler