euske / pdfminer

Python PDF Parser (Not actively maintained). Check out pdfminer.six.
https://github.com/pdfminer/pdfminer.six
MIT License
5.24k stars 1.13k forks source link

PDFDocument is slow, any way to speed it up? #283

Open typhoon71 opened 4 years ago

typhoon71 commented 4 years ago

I'm currently doing this:

fp = open(pdf_fpath, 'rb')
parser = PDFParser(fp)
doc = PDFDocument(parser)

With the pdfs I'm working with, PDFDocument is really too slow. Is there any way to speed it up?

I see that there's often the suggestion to "giving -n option which turns off automatic layout analysis.", but I have no idea how to do that from python.

I "only" need to read xrefs and objects/streams, I don't care how the pdf would be rendered/pages. Can anyone help please? Thanks.

typhoon71 commented 4 years ago

From that example:

parser = PDFParser(in_file)
doc = PDFDocument(parser)

was exactly where I timed the slowdown.

 doc = PDFDocument(parser)

is time consumnig, like 99%.

If I understand your suggestion, I should make my own version of PDFDocument? One that does not analyze the layout?

No time for it right now, but maybe I could in the future.

Actually I hoped in some kind of switch for PDFDocument... but I found nothing.

pietermarsman commented 4 years ago

I'm sorry. My comment was short, not clear and partially wrong. I've deleted it.

Layout analysis is not used during the parsing of the PDF. It is only used when you interpret the document, with e.g. TextConverter of HTMLConverter.

Could you share an example PDF that PDFDocument is really slow on?

typhoon71 commented 4 years ago

Sadly I can't share those PFDs. The thing is that once I got PDFDocument initialized, everything else is really fast. Maybe it's decompressing the PDF, that could be slowing it.

hiDaDeng commented 2 years ago
import fitz

filepath = "C:\\user\\docs\\aPDFfile.pdf"

text = ''
with fitz.open(filepath ) as doc:
    for page in doc:
        text+= page.get_text()
print(text)