search

esmero / strawberryfield

A Field of strawberries
GNU Lesser General Public License v3.0
10 stars 5 forks source link

Expand Basic Metadata extracted for Files, in specific PDF and Clean EXIF History #103

Closed DiegoPino closed 1 month ago

DiegoPino commented 4 years ago

What, more file mangling? yes!

This is a continuation of #86 and #87 which was merged.

The need

Better understanding of what is inside a PDF

Right now we are just getting general PDFinfo (single first page), which means in our metadata we only keep number of Pages (good) and IF even , a single page Dimension. Not cool for Rare books, complex displays in general and too simplistic to be honest when dealing with a IIIF Manifest generation we want to allow to work on Mirador and the Book reader since our implementation (also simplistic) of https://github.com/mozilla/pdf.js is a bit slow on large super large PDFs. 👀 @tomadams re:your email today

Solution. Simple. Get more Metadata. How?

Run PDF Info twice: 1.- get the pages as we do now 2.- then use the -f and -l arguments to get all the dimensions for all pages. Store that into an array and add to the JSON. 1000 pages, 1000 entries? May need to think about that but seems feasible, but could also go directly into SOLR same way we expect Text extraction, HOCR and entity extraction would happen per page (one Solr doc per page).

Use that data in the manifest and also rewrite our manifests. The one we have in play.archipelago.nyc is passing the IIIF V2 tests correctly, we need the same for IIIF V3.

Clean EXIF

Ok, still confused about this. @alliomeria may know better. Will put two Examples here: first one clean EXIF

https://play.archipelago.nyc/do/f4a4c6ee-4ce9-4b4c-8704-e8057bad0a7d

{
 "flv:exif": {
                "ISO": 100,
                "Flash": "No Flash",
                "Model": "RICOH THETA S",
                "Aperture": 2,
                "FileSize": "2.8 MB",
                "MIMEType": "image\/jpeg",
                "ImageSize": "5376x2688",
                "Sharpness": "Normal",
                "ColorSpace": "sRGB",
                "ImageWidth": 5376,
                "XMPToolkit": "RICOH THETA for iOS 2.14.0",
                "FocalLength": "1.3 mm",
                "ImageHeight": 2688,
                "GPSVersionID": "2.3.0.0",
                "MeteringMode": "Multi-segment",
                "ShutterSpeed": "1\/6400",
                "WhiteBalance": "Auto",
                "ProjectionType": "equirectangular",
                "GPSImgDirection": 270,
                "PoseRollDegrees": 0,
                "DateTimeOriginal": "2020:07:02 17:25:15",
                "PosePitchDegrees": 0,
                "UsePanoramaViewer": true,
                "GPSImgDirectionRef": "True North",
                "PoseHeadingDegrees": 0,
                "FullPanoWidthPixels": 5376,
                "CroppedAreaTopPixels": 0,
                "ExposureCompensation": 0,
                "FullPanoHeightPixels": 2688,
                "CroppedAreaLeftPixels": 0,
                "CroppedAreaImageWidthPixels": 5376,
                "CroppedAreaImageHeightPixels": 2688
            }
}

Unclean (see duplication because of changes history in the second PDF) http://ec2-184-73-148-144.compute-1.amazonaws.com/do/018744ea-1d99-4d71-bd93-6cd402a82d74

```JSON PRESS HERE TO SEE ALL! { "flv:exif": { "Title": "Basic RGB", "Format": "application\/pdf", "NPages": 1, "FileSize": "1934 kB", "FontFace": [ "Regular", "Regular" ] } } ``` ```JSON { "flv:exif": { "Title": "Basic RGB", "Format": "application\/pdf", "NPages": 1, "FileSize": "1934 kB", "FontFace": [ "Regular", "Regular" ], "FontName": [ "BebasNeue-Regular", "MyriadPro-Regular" ], "FontType": [ "Open Type", "Open Type" ], "MIMEType": "application\/pdf", "Producer": "Adobe PDF library 10.01", "PageCount": 1, "CreateDate": "2019:11:01 17:16:50-04:00", "DocumentID": "xmp.did:e88490b4-4350-2243-9e6a-e0e8a9092ec9", "FontFamily": [ "Bebas Neue", "Myriad Pro" ], "InstanceID": "uuid:fac62424-48d5-4b85-84fd-49beb49d517c", "Linearized": "No", "ModifyDate": "2019:11:01 21:58:38-04:00", "PDFVersion": 1.5, "PlateNames": [ "Cyan", "Magenta", "Yellow", "Black" ], "XMPToolkit": "Adobe XMP Core 5.6-c145 79.163499, 2018\/08\/13-16:40:22 ", "CreatorTool": "Adobe Illustrator CC 23.1 (Windows)", "FontVersion": [ "Version 2.000;PS 002.000;hotconv 1.0.88;makeotf.lib2.5.64775", "Version 2.106;PS 2.000;hotconv 1.0.70;makeotf.lib2.5.58329" ], "HistoryWhen": [ "2019:10:30 12:21:32-04:00", "2019:11:01 17:16:51-04:00" ], "FontFileName": [ 13407, "MyriadPro-Regular.otf" ], "MaxPageSizeH": 28, "MaxPageSizeW": 42, "MetadataDate": "2019:11:01 21:58:38-04:00", "FontComposite": [ false, false ], "HistoryAction": [ "saved", "saved" ], "CreatorVersion": 23, "HistoryChanged": [ "\/", "\/" ], "RenditionClass": "proof:pdf", "StartupProfile": "Basic RGB", "ThumbnailWidth": 256, "MaxPageSizeUnit": "Inches", "SwatchGroupName": [ "Default Swatch Group", "Cold", "Grays" ], "SwatchGroupType": [ 0, 0, 0 ], "ThumbnailFormat": "JPEG", "ThumbnailHeight": 172, "ContainerVersion": 11, "ManifestLinkForm": [ "EmbedByReference", "EmbedByReference" ], "HistoryInstanceID": [ "xmp.iid:615189d1-95dc-e64c-b838-2a31d901c875", "xmp.iid:e88490b4-4350-2243-9e6a-e0e8a9092ec9" ], "SwatchColorantRed": [ 255, 0, 255, 255, 0, 0, 0, 255, 192, 236, 240, 246, 250, 251, 216, 139, 57, 0, 0, 34, 0, 41, 0, 46, 27, 102, 146, 157, 211, 236, 198, 152, 115, 83, 197, 165, 139, 117, 96, 66, 101, 130, 185, 0, 26, 51, 77, 102, 128, 152, 178, 203, 229, 241 ], "EmbeddedImageWidth": 381, "OriginalDocumentID": "uuid:9E3E5C9A8C81DB118734DB58FDDE4BA7", "SwatchColorantBlue": [ 255, 0, 0, 0, 0, 255, 255, 255, 45, 36, 36, 30, 59, 33, 33, 63, 74, 69, 55, 115, 156, 225, 187, 145, 100, 144, 142, 93, 90, 121, 152, 117, 87, 65, 109, 82, 57, 36, 19, 11, 207, 196, 200, 0, 26, 51, 77, 102, 128, 152, 178, 203, 229, 241 ], "SwatchColorantMode": [ "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB", "RGB" ], "SwatchColorantType": [ "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS", "PROCESS" ], "EmbeddedImageFilter": "FlateDecode", "EmbeddedImageHeight": 602, "HasVisibleOverprint": false, "IngredientsFilePath": [ "C:\\Users\\clee4\\Downloads\\mora_2018_figures\\Photos to Share\\CAD Images\\20191101-DSC00067.jpg", "C:\\Users\\clee4\\Downloads\\mora_2018_figures\\Photos to Share\\Poster and Paper Figures\\coral-reef-drawing-10.png" ], "SwatchColorantGreen": [ 255, 0, 0, 255, 255, 255, 0, 0, 39, 28, 90, 146, 175, 237, 223, 197, 180, 145, 104, 180, 168, 170, 113, 49, 20, 45, 39, 0, 20, 30, 177, 133, 99, 71, 155, 124, 98, 76, 56, 33, 199, 138, 154, 0, 26, 51, 77, 102, 128, 152, 178, 203, 229, 241 ], "HistorySoftwareAgent": [ "Adobe Illustrator CC 22.1 (Windows)", "Adobe Illustrator CC 23.1 (Windows)" ], "DerivedFromDocumentID": "xmp.did:f6f8d79a-3268-9b47-b8ca-5ae8fc53d04a", "DerivedFromInstanceID": "xmp.iid:f6f8d79a-3268-9b47-b8ca-5ae8fc53d04a", "IngredientsDocumentID": [ "xmp.did:24918461-5358-463f-8f02-8a25bbc0f753", "adobe:docid:photoshop:80092d49-6ba7-f649-b041-a6d0af913b7f" ], "IngredientsInstanceID": [ "xmp.iid:24918461-5358-463f-8f02-8a25bbc0f753", "xmp.iid:c964d0b5-30e9-854d-9422-d12453198b63" ], "HasVisibleTransparency": true, "EmbeddedImageColorSpace": [ "DeviceRGB", "Indexed", "DeviceRGB", 1, "DeviceRGB" ], "SwatchColorantSwatchName": [ "White", "Black", "RGB Red", "RGB Yellow", "RGB Green", "RGB Cyan", "RGB Blue", "RGB Magenta", "R=193 G=39 B=45", "R=237 G=28 B=36", "R=241 G=90 B=36", "R=247 G=147 B=30", "R=251 G=176 B=59", "R=252 G=238 B=33", "R=217 G=224 B=33", "R=140 G=198 B=63", "R=57 G=181 B=74", "R=0 G=146 B=69", "R=0 G=104 B=55", "R=34 G=181 B=115", "R=0 G=169 B=157", "R=41 G=171 B=226", "R=0 G=113 B=188", "R=46 G=49 B=146", "R=27 G=20 B=100", "R=102 G=45 B=145", "R=147 G=39 B=143", "R=158 G=0 B=93", "R=212 G=20 B=90", "R=237 G=30 B=121", "R=199 G=178 B=153", "R=153 G=134 B=117", "R=115 G=99 B=87", "R=83 G=71 B=65", "R=198 G=156 B=109", "R=166 G=124 B=82", "R=140 G=98 B=57", "R=117 G=76 B=36", "R=96 G=56 B=19", "R=66 G=33 B=11", "C=56 M=0 Y=20 K=0", "C=51 M=43 Y=0 K=0", "C=26 M=41 Y=0 K=0", "R=0 G=0 B=0", "R=26 G=26 B=26", "R=51 G=51 B=51", "R=77 G=77 B=77", "R=102 G=102 B=102", "R=128 G=128 B=128", "R=153 G=153 B=153", "R=179 G=179 B=179", "R=204 G=204 B=204", "R=230 G=230 B=230", "R=242 G=242 B=242" ], "DerivedFromRenditionClass": "proof:pdf", "ManifestReferenceFilePath": [ "C:\\Users\\clee4\\Downloads\\mora_2018_figures\\Photos to Share\\CAD Images\\20191101-DSC00067.jpg", "C:\\Users\\clee4\\Downloads\\mora_2018_figures\\Photos to Share\\Poster and Paper Figures\\coral-reef-drawing-10.png" ], "ManifestReferenceDocumentID": [ "xmp.did:24918461-5358-463f-8f02-8a25bbc0f753", "adobe:docid:photoshop:80092d49-6ba7-f649-b041-a6d0af913b7f" ], "ManifestReferenceInstanceID": [ "xmp.iid:24918461-5358-463f-8f02-8a25bbc0f753", "xmp.iid:c964d0b5-30e9-854d-9422-d12453198b63" ], "DerivedFromOriginalDocumentID": "uuid:9E3E5C9A8C81DB118734DB58FDDE4BA7" } } ````

Question is: Do we de-dup?, do we simply strip from EXIF a list of offenders? I mean i love the idea of indexing in Solr the Colorswatches, but its a lot, like really too much?

alliomeria commented 4 years ago

Ok, still confused about this. @alliomeria may know better.

@DiegoPino I know not better, only that there is/can be considerable variance in EXIF metadata, even for supposedly ISO-standard-compliant PDFs. :)

Question is: Do we de-dup?, do we simply strip from EXIF a list of offenders? I mean i love the idea of indexing in Solr the Colorswatches, but its a lot, like really too much?

@DiegoPino De-duping could be helpful for those history change redundancies, but also don't think it'd hurt much to let be (has a human-readable "this document was changed" vibe). I agree that while cool, the SwatchColorantXX sprawl is too much for Solr. Feel that the need for Solr indexing for this would be an extremely niche (perhaps interesting?) situation/use case.

Speaking of use cases, regarding the PDF info extraction of every page, can certainly see this being important and useful for complex displays as you mention. Also thinking of objects with IIIF annotations down the road, wondering if this could be relevant for potential instances of multipage PDFs with annotations throughout.

DiegoPino commented 4 years ago

Thanks @alliomeria So i have 3 choices now 1.- Usign PDFInfo and extracting first page numbers and then PDFInfo again (command shared) Media boxes for every page 

pdfinfo 3pagesofdifferentsizes.pdf -f 1 -l 3 -box 
Title:     /root/images2/HJM01_000176/HJM01_000176_001.pdf
Producer:    ImageMagick 6.2.4 11/01/06 Q16 http://www.imagemagick.org
CreationDate:  Fri Jan 5 07:08:37 2007 EST
ModDate:    Fri Jan 5 07:08:37 2007 EST
Tagged:     no
UserProperties: no
Suspects:    no
Form:      none
JavaScript:   no
Pages:     3
Encrypted:   no
Page  1 size: 348 x 554 pts
Page  1 rot: 0
Page  2 size: 365 x 291 pts
Page  2 rot: 0
Page  3 size: 365 x 289 pts
Page  3 rot: 0
Page  1 MediaBox:   0.00   0.00 1935.00 3077.00
Page  1 CropBox:   0.00   0.00  348.00  554.00
Page  1 BleedBox:   0.00   0.00  348.00  554.00
Page  1 TrimBox:   0.00   0.00  348.00  554.00
Page  1 ArtBox:    0.00   0.00  348.00  554.00
Page  2 MediaBox:   0.00   0.00 2030.00 1617.00
Page  2 CropBox:   0.00   0.00  365.00  291.00
Page  2 BleedBox:   0.00   0.00  365.00  291.00
Page  2 TrimBox:   0.00   0.00  365.00  291.00
Page  2 ArtBox:    0.00   0.00  365.00  291.00
Page  3 MediaBox:   0.00   0.00 2028.00 1605.00
Page  3 CropBox:   0.00   0.00  365.00  289.00
Page  3 BleedBox:   0.00   0.00  365.00  289.00
Page  3 TrimBox:   0.00   0.00  365.00  289.00
Page  3 ArtBox:    0.00   0.00  365.00  289.00
File size:   47303172 bytes
Optimized:   no
PDF version:  1.3 (edited)

Second choice is the somewhat slower identify and gives me some format options so i can split and built back

identify -format 'format:%m|width:%w|height:%h|exif_orientation:%[EXIF:Orientation]@' -quiet 3pagesofdifferentsizes.pdf

format:PDF|width:1935|height:3077|exif_orientation:@format:PDF|width:2030|height:1617|exif_orientation:@format:PDF|width:2028|height:1605|exif_orientation:@

And the third one is using actually Drupal directly and its wrapper. Issue is even if the machinery is quite sophisticated its also limiting and easy to break everything when doing normal changes any CMS site builder could do.

It would imply Drupal Imagemagick and Drupal FMD

Any option implies just changes for now in SBF module and in a single file but makes me believe its about time we move to SBF Runners starting from 1.0.1

alliomeria commented 4 years ago

@DiegoPino (from my humble perspective) The second choice seems the most practical and straightforward. For this choice, how slow is slow (or too slow)?

DiegoPino commented 4 years ago

@alliomeria i agree. It is significantly slower in my local machine (5 seconds for a PDF on identify v/s 400 ms using pdfinfo), but then Docker runs very slow on OSX when doing native File system access as we do in our deployment and Imagemagick is heavier because it is really getting way more info but we are just formatting what we need back.

Pushed the code here now https://github.com/esmero/strawberryfield/tree/ISSUE-101 (need to rebase to i can merge into 1.0.0-RC1)

And the result is this:

 "urn:uuid:ac7fc929-a45e-4abc-9ff4-a6c35ec16c2f": {
            "url": "s3:\/\/7d6\/application-001-ac7fc929-a45e-4abc-9ff4-a6c35ec16c2f.pdf",
            "name": "a3pagepdf.pdf",
            "tags": [],
            "type": "Document",
            "dr:fid": 2357,
            "dr:for": "documents",
            "dr:uuid": "ac7fc929-a45e-4abc-9ff4-a6c35ec16c2f",
            "checksum": "7d644b3e545f8efc2704cafb97c34ffc",
            "flv:exif": {
                "Title": "\/root\/images\/a3pagepdf.pdf",
                "FileSize": "45 MB",
                "MIMEType": "application\/pdf",
                "Producer": "ImageMagick 6.2.4 11\/01\/06 Q16 http:\/\/www.imagemagick.org",
                "PageCount": 3,
                "CreateDate": "2007:01:05 12:08:37",
                "Linearized": "No",
                "ModifyDate": "2007:01:05 12:08:37",
                "PDFVersion": 1.3,
                "EmbeddedImageWidth": 1935,
                "EmbeddedImageFilter": "ASCII85Decode",
                "EmbeddedImageHeight": 3077,
                "EmbeddedImageColorSpace": [
                    "DeviceRGB"
                ]
            },
            "sequence": 2,
            "flv:pronom": {
                "label": "Acrobat PDF 1.3 - Portable Document Format",
                "mimetype": "application\/pdf",
                "pronom_id": "info:pronom\/fmt\/17",
                "detection_type": "signature"
            },
            "dr:mimetype": "application\/pdf",
            "crypHashFunc": "md5",
            "flv:identify": {
                "1": {
                    "width": "1935",
                    "format": "PDF",
                    "height": "3077",
                    "orientation": "Undefined"
                },
                "2": {
                    "width": "2030",
                    "format": "PDF",
                    "height": "1617",
                    "orientation": "Undefined"
                },
                "3": {
                    "width": "2028",
                    "format": "PDF",
                    "height": "1605",
                    "orientation": "Undefined"
                }
            }
        }
    },

now inside flv:identify we get each page characterized with its own size. Means we can now use that data while building the IIIF manifest via the twig template.

DiegoPino commented 4 years ago

Short note: "orientation" here allows me to read back metadata hints on rotation. Since those can be on EXIF and/or TIFF (XMP) tags, orientation can keep track/extract both options transparently.

That can then be used in Cantaloupe to force a certain rotation (RE: the NYSPERSONALHISTORY use case where images are rotated in the phone but cantaloupe fails to acknowledge)

DiegoPino commented 1 month ago

I don't think touching Technical metadata is an archipelago concern. We might do more harm if we attempt this. EXIF is also a moving target. Closing as won't implement