JorjMcKie
commented
6 years ago This is not specific to PyMuPDF: If you use the mudraw cli utility of MuPDF to print the text, you get this:
200
200
Customer information
Customer information
Customer
Customer
information
information
Customer information
Customer information ................ 200
200
Declaration of conformity ......... 200
Vehicle data recording and pri‐
Vehicle data recording and pri‐
vacy
vacy ........................................... 202
202
Event data recorders ............... 202
Radio Frequency Identification
(RFID) ..................................... 202
Customer information
Customer information
Declaration of conformity
Declaration of conformity
Transmission systems
Transmission systems
This vehicle has systems that
transmit and/or receive radio waves
subject to Directive 1999/5/EC.
These systems are in compliance
with the essential requirements and
other relevant provisions of
Directive 1999/5/EC. Copies of the
original Declarations of Conformity
can be obtained on our website.
Radar systems
Radar systems
Country-specific Declarations of
Conformity for radar systems are
shown on the following page:
which is exactly the same - except that PyMuPDF in contrast does not insert extra line breaks for lines within the same block. So this is a peculiarity of the PDF itself: it does contains the same word multiple times. To confirm, I used one of my utilities that analyzes a PDF page layout, drawing a rectangle around each block and then inserting the block's text into it. I am getting this:
So, what can you do to detect if words occur multiple times? Here is a little script that does this:
import fitz
doc = fitz.open(...)
page = doc[201]
from operator import itemgetter
words = page.getTextWords() # get all words info on page
words.sort(key = itemgetter(1,0)) # sort by vertical, horizontal position
h = words[0][1] # vertical position of top-left word
oldrect = fitz.Rect() # control if word is in same rectangle
lines = [] # all the re-created lines go here
line = "" # a fresh re-created line
for w in words:
newrect = fitz.Rect(w[:4]) # rectangle of word
if newrect == oldrect: # same as old one?
continue # skip
oldrect = newrect # store our rect
y = w[1] # vertical coord of word
word = w[4] # the word text
if y == h: # on same line?
line += word # append word
line += " " # append a space
else: # start new line
lines.append(line)
line = word + " "
h = y
lines.append(line) # append unfinished line
for line in lines: # print the re-created lines
print(line)
It's output:
200 Customer information
Customer
Customer information
information
Declaration of conformity
Transmission systems
Customer information ................ 200
This vehicle has systems that
Declaration of conformity ......... 200
transmit and/or receive radio waves
subject to Directive 1999/5/EC.
Vehicle data recording and pri‐
These systems are in compliance
vacy ........................................... 202
with the essential requirements and
Event data recorders ............... 202
other relevant provisions of
Radio Frequency Identification
Directive 1999/5/EC. Copies of the
(RFID) ..................................... 202
original Declarations of Conformity
can be obtained on our website.
Radar systems
Country-specific Declarations of
Conformity for radar systems are
shown on the following page:Can obviously still be improved ... maybe by combining it with the layout analyzer mentioned above, or whatever.
Please tell me your platform and PyMuPDF version number for the records.
Hugloss
Note that the physical page is 123 (1-based, 122 0-based), whereas the logical page is 121 (document's own numbering scheme).
This is the output from PyMuPDF:
You will find
using this pdf: https://www.opel.ie/content/dam/opel/ireland/owners/manuals/pdf/vivaro/om_vivaro_kta-2769_2-en_eu_my16_ed0415_5_en_gb.pdf
for example the result for page 200:
200 200 Customer information Customer information Customer Customer information information Customer information Customer information ................ 200 200 Declaration of conformity ......... 200 Vehicle data recording and pri‐ Vehicle data recording and pri‐ vacy vacy ........................................... 202 202 Event data recorders ............... 202 Radio Frequency Identification (RFID) ..................................... 202 Customer information Customer information Declaration of conformity Declaration of conformity Transmission systems Transmission systems This vehicle has systems that transmit and/or receive radio waves subject to Directive 1999/5/EC. These systems are in compliance with the essential requirements and other relevant provisions of Directive 1999/5/EC. Copies of the original Declarations of Conformity can be obtained on our website. Radar systems Radar systems Country-specific Declarations of Conformity for radar systems are shown on the following page:has fault text at row 7,8,9
The correct text should be