search

numbersprotocol / pyc2pa

Python implementation of C2PA: Coalition for Content Provenance and Authenticity.
GNU General Public License v3.0
36 stars 3 forks source link

Doing multi-injection multiple times causes the injected image to be corrupted #12

Open shc261392 opened 3 years ago

shc261392 commented 3 years ago

There are 2 scenarios of this issue. I'm not sure if they share the same root cause, but since the scenarios are similar I'll include both here.

Scenario 1

When using the modified example script, leaving the thumbnail_bytes empty will corrupt the 3rd multi-injected image, and no exception is raised during the process.

Modified example script

This is a minimal modification base on https://github.com/numbersprotocol/starling-cai/blob/master/utils/starling_multiple_injection.py to reproduce the issue.

Run the script with python3 <script-file> <jpeg-image-to-be-injected>

#!/usr/bin/python3
#
# Starling Hello World Example for multi-injection.
#
# This program will inject 3 CAI Stores.
#
# Usage
#     $ python3 starling_multiple_injection.py <image-filepath>
#
# Verify
#     1. Go to https://verify.contentauthenticity.org/inspect
#     2. Drag and drop the generated image.
#     3. The webpage will show the CAI information.

import os
import sys

from cai.jumbf import json_to_bytes
from cai.starling import Starling

photo_filename = sys.argv[1]
# thumbnail_filename = sys.argv[1].replace('.jpg', '-thumbnail.jpg')

photo_bytes = open(photo_filename, 'rb').read()
# thumbnail_bytes = open(thumbnail_filename, 'rb').read()

location = ''
owner = ''
thumbnail_bytes = b''
public_key = ''
media_hash = ''
timestamp = ''
recorder1 = ''
recroder2 = ''
recorder3 = ''

metadata = [
    {
        'claim': {
            'store_label': 'cb.Authmedia_1',
            'recorder': recorder1,
        },
        'assertions': {
            'adobe.asset.info': {
                'type': '.json',
                'data_bytes': json_to_bytes({
                    'title': photo_filename
                })
            },
            'cai.location.broad': {
                'type': '.json',
                'data_bytes': json_to_bytes({
                    'location': location
                })
            },
            'cai.rights': {
                'type': '.json',
                'data_bytes': json_to_bytes({
                    'copyright': owner
                })
            },
            'cai.claim.thumbnail.jpg.jpg': {
                'type': '.jpg',
                'data_bytes': thumbnail_bytes
            },
            'cai.acquisition.thumbnail.jpg.jpg': {
                'type': '.jpg',
                'data_bytes': thumbnail_bytes
            },
            'starling.integrity.json': {
                'type': '.json',
                'data_bytes': json_to_bytes({
                    'starling:PublicKey': public_key,
                    'starling:MediaHash': media_hash,
                    'starling:MediaKey': '',
                    'starling:CaptureTimestamp': timestamp,
                })
            }
        }
    },
    {
        'claim': {
            'store_label': 'cb.IOTAIntegrityChain_2',
            'recorder': recroder2,
        },
        'assertions': {
            'adobe.asset.info': {
                'type': '.json',
                'data_bytes': json_to_bytes({
                    'title': photo_filename
                })
            },
            'cai.location.broad': {
                'type': '.json',
                'data_bytes': json_to_bytes({
                    'location': location
                })
            },
            'cai.rights': {
                'type': '.json',
                'data_bytes': json_to_bytes({
                    'copyright': owner
                })
            },
            'cai.claim.thumbnail.jpg.jpg': {
                'type': '.jpg',
                'data_bytes': thumbnail_bytes
            },
            'cai.acquisition.thumbnail.jpg.jpg': {
                'type': '.jpg',
                'data_bytes': thumbnail_bytes
            },
            'starling.integrity.json': {
                'type': '.json',
                'data_bytes': json_to_bytes({
                    'starling:PublicKey': public_key,
                    'starling:MediaHash': media_hash,
                    'starling:MediaKey': '',
                    'starling:CaptureTimestamp': timestamp
                })
            }
        }
    },
    {
        'claim': {
            'store_label': 'cb.ThunderCoreNFTChain_3',
            'recorder': recorder3,
        },
        'assertions': {
            'adobe.asset.info': {
                'type': '.json',
                'data_bytes': json_to_bytes({
                    'title': photo_filename
                })
            },
            'cai.location.broad': {
                'type': '.json',
                'data_bytes': json_to_bytes({
                    'location': location
                })
            },
            'cai.rights': {
                'type': '.json',
                'data_bytes': json_to_bytes({
                    'copyright': owner
                })
            },
            'cai.claim.thumbnail.jpg.jpg': {
                'type': '.jpg',
                'data_bytes': thumbnail_bytes
            },
            'cai.acquisition.thumbnail.jpg.jpg': {
                'type': '.jpg',
                'data_bytes': thumbnail_bytes
            },
            'starling.integrity.json': {
                'type': '.json',
                'data_bytes': json_to_bytes({
                    'starling:PublicKey': public_key,
                    'starling:MediaHash': media_hash,
                    'starling:CaptureTimestamp': timestamp,
                })
            }
        }
    },
]

# 1st CAI injection: Authmedia
# 2nd CAI injection: IOTA
# 3rd CAI injection: ThunderCore
for i in range(3):
    starling = Starling(photo_bytes,
                        photo_filename,
                        metadata[i]['assertions'],
                        metadata[i]['claim']['store_label'],
                        metadata[i]['claim']['recorder'],
                        '',
                        '')
    photo_bytes = starling.cai_injection()

# Save to file
fname, fext = os.path.splitext(photo_filename)
fpath = fname + '-cai-cai-cai' + fext
with open(fpath, 'wb') as f:
    f.write(photo_bytes)

Corrupted image

bread-cai-cai-cai

Scenario 2

This thirdly injected image has valid thumbnail_bytes value (it can be seen on the Verify site), but it is still corrupted. I haven't figured out the reason yet.

Corrupted image

2QH7lHH2KtbwHbsUqlQn_bread_cai__3__

bafu commented 3 years ago

Verified bread-cai-cai-cai by jp2file and got a thumbnail-relaed error

$ python3 codestream_parser/jp2file.py ~/codes/starling-cai/utils/james-cai-cai-cai.jpeg 
###############################################################                                      # JP2 file format log file generated by jp2file.py            ################################################################
                                                                                                     0       : New marker: SOI (Start of image)                                                          
                                                                                                     2       : New marker: APP1 (Application marker #1)                                                                   
                                                                                                     2459    : New marker: APP11 (JPEG XT Extension Marker)                                                  
    0       : Sub Box: "jumb" JUMBF Box                                                                    8       : Sub Box: "jumd"         JUMBF Description box        TYPE: 63 61 63 62 00 11 00 10 80 00 00 aa 00 38 9b 71
        TOGGLES: 0b11                                                                                        LABEL: cai                                
        No ID                                     
        No Signature                                                                                                                                                                                      
      37      : Sub Box: "jumb" JUMBF Box                                                            
        45      : Sub Box: "jumd"           JUMBF Description box                                              TYPE: 63 61 73 74 00 11 00 10 80 00 00 aa 00 38 9b 71
          TOGGLES: 0b11                           
          LABEL: cb.Authmedia_1                                                                                No ID                                   
          No Signature                                                                               
                                                                                                             85      : Sub Box: "jumb" JUMBF Box                                                          
          93      : Sub Box: "jumd"             JUMBF Description box
            TYPE: 63 61 61 73 00 11 00 10 80 00 00 aa 00 38 9b 71                                                TOGGLES: 0b11            
            LABEL: cai.assertions                 
            No ID                                                                                    
            No Signature                                                                             

          133     : Sub Box: "jumb" JUMBF Box
            141     : Sub Box: "jumd"               JUMBF Description box
              TYPE: 6a 73 6f 6e 00 11 00 10 80 00 00 aa 00 38 9b 71
              TOGGLES: 0b11
              LABEL: adobe.asset.info
              No ID
              No Signature

            183     : Sub Box: "json" JSON box
              Data:
              {"title":"bread.jpg"}

          212     : Sub Box: "jumb" JUMBF Box
            220     : Sub Box: "jumd"               JUMBF Description box
              TYPE: 6a 73 6f 6e 00 11 00 10 80 00 00 aa 00 38 9b 71
              TOGGLES: 0b11
              LABEL: cai.location.broad
              No ID
              No Signature

            264     : Sub Box: "json" JSON box
              Data:
              {"location":""}

          287     : Sub Box: "jumb" JUMBF Box
            295     : Sub Box: "jumd"               JUMBF Description box
              TYPE: 6a 73 6f 6e 00 11 00 10 80 00 00 aa 00 38 9b 71
              TOGGLES: 0b11
              LABEL: cai.rights
              No ID
              No Signature

            331     : Sub Box: "json" JSON box
              Data:
              {"copyright":""}

          355     : Sub Box: "jumb" JUMBF Box
            363     : Sub Box: "jumd"               JUMBF Description box
              TYPE: 65 79 d6 fb db a2 44 6b b2 ac 1b 82 fe eb 89 d1
              TOGGLES: 0b11
              LABEL: cai.claim.thumbnail.jpg.jpg
              No ID
              No Signature

            416     : Sub Box: "jp2c" Codestream box
Traceback (most recent call last):
  File "codestream_parser/jp2file.py", line 2334, in <module>
    jpg.stream_parse(file,0)
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jpgcode
stream.py", line 439, in stream_parse
    self.parse_table()
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jpgcode
stream.py", line 370, in parse_table
    self.parse_APP(ordw(self.buffer) - 0xffe0)
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jpgcode
stream.py", line 323, in parse_APP
    box.parse(self.superhook)
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2box.
py", line 156, in parse
    hook(self,id,"%d" % length)
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2file
.py", line 2133, in superbox_hook
    parse_superbox(box,"JUMBF Box")
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2file
.py", line 2043, in parse_superbox
    box.parse(superbox_hook)
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2box.
py", line 156, in parse
    hook(self,id,"%d" % length)
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2file
.py", line 2133, in superbox_hook
    parse_superbox(box,"JUMBF Box")
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2file
.py", line 2043, in parse_superbox
    box.parse(superbox_hook)
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2box.
py", line 156, in parse
    hook(self,id,"%d" % length)
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2file
.py", line 2133, in superbox_hook
    parse_superbox(box,"JUMBF Box")
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2file
.py", line 2043, in parse_superbox
    box.parse(superbox_hook)
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2box.
py", line 156, in parse
    hook(self,id,"%d" % length)
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2file
.py", line 2133, in superbox_hook
    parse_superbox(box,"JUMBF Box")
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2file
.py", line 2043, in parse_superbox
    box.parse(superbox_hook)
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2box.
py", line 156, in parse
    hook(self,id,"%d" % length)
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2file
.py", line 2079, in superbox_hook
    cs.stream_parse(box.infile,box.offset)
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2code
stream.py", line 194, in stream_parse
    self.load_buffer(file)
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2code
stream.py", line 185, in load_buffer
    self.load_marker(file,marker)
  File "/home/bafu/codes/codestream-parser/venv/lib/python3.8/site-packages/codestream_parser/jp2code
stream.py", line 158, in load_marker
    mrk    = ((ord(marker[0]) << 8) +
TypeError: ord() expected string of length 1, but int found
bafu commented 3 years ago

Verified "good thumb image" by jp2file and can pass verification.

bafu commented 3 years ago

Can not reproduce this issue by using James' example: python3 james_multiple_injection.py doge-meets-bread.jpeg

Note

  1. When generating a thumbnail, use the original image instead of an image containing CAI metadata. It will cause an unknown issue if a thumbnail contains CAI metadata.
  2. In my environment, if I create a thumbnail from a CAI-injected image, the thumbnail will still contains the CAI metadata.

Source and Workable Multi-Injection Images

Source image

Workable multi-injection image created by James' example w/ the patch

--- a/james_multiple_injection.py    2021-04-27 11:52:57.907932857 +0800
+++ b/james_multiple_injection.py 2021-04-27 11:46:59.239599598 +0800
@@ -20,14 +20,14 @@

 photo_filename = sys.argv[1]
-#thumbnail_filename = sys.argv[1].replace('.jpg', '-thumbnail.jpg')
+thumbnail_filename = sys.argv[1].replace('.jpg', '-thumbnail.jpg')

 photo_bytes = open(photo_filename, 'rb').read()
-#thumbnail_bytes = open(thumbnail_filename, 'rb').read()
+thumbnail_bytes = open(thumbnail_filename, 'rb').read()

 location = ''
 owner = ''
-thumbnail_bytes = b''
+#thumbnail_bytes = b''
 public_key = ''
 media_hash = ''
 timestamp = ''

The thumbnail is generated by convert -resize 50% doge-meets-bread.jpeg doge-meets-bread-thumbnail.jpeg

doge-meets-bread-cai-cai-cai-james

Workable multi-injection image created by starling_multiple_injection.py

shc261392 commented 3 years ago

bread

bread-thumbnail

A corrupted image can be created by using the above script with this image and this thumbnail

bread-cai-cai-cai

bafu commented 3 years ago

Manually injected by using the raw image and thumbnail in the comment above, and the injected image below (bread-cai-cai-cai.jpeg) is normal: bread-cai-cai-cai

$ python3 james_multiple_injection.py bread.jpeg

$ sha256sum bread*
af30fc972c6538fb3fc14f940cf027df588345c8456c530f2f14ca85d2e6544e  bread-cai-cai-cai.jpeg
1493250b88cfda3c6879732485789a0b799a62367e2ff2d8a0e8927ffa75bb6b  bread.jpeg
5412d7806f0579ead0b97ada042cbed5871aa0208bfa39b8b8fad7004638c24e  bread-thumbnail.jpeg
bafu commented 3 years ago

I can reproduce this issue on storage-backend instance.

steps to reproduce

  1. created a clean venv
  2. ran testing script
  3. got the same broken image with James'

Both AWS venv and my local venv have the same deps

diff -Naru venv-deps-aws.txt venv-deps-bofu.txt 
--- venv-deps-aws.txt   2021-04-28 13:00:53.607844061 +0800
+++ venv-deps-bofu.txt  2021-04-28 13:01:44.053226163 +0800
@@ -1,7 +1,7 @@
 asn1crypto==1.4.0
 attrs==20.3.0
 bcrypt==3.2.0
-cai @ file:///home/ubuntu/cai/cai-1.3.0-py3-none-any.whl
+cai @ file:///home/bafu/Downloads/cai-1.3.0-py3-none-any.whl
 certifi==2020.12.5
 cffi==1.14.5
 chardet==4.0.0

Action

  1. James will verify on a new Ubuntu 20.04 instance.
  2. Bofu will verify on DLC which is 20.04, too.
bafu commented 3 years ago

DLC server needs to install the system dependencies

Got the error when install the cai module in venv:

  x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.8/src/exiv2wrapper.o build/temp.linux-x86_64-3.8/src/exiv2wrapper_python.o -lboost_python38 -lexiv2 -o build/lib.linux-x86_64-3.8/libexiv2python.cpython-38-x86_64-linux-gnu.so
  /usr/bin/ld: cannot find -lboost_python38
  collect2: error: ld returned 1 exit status
  error: command 'x86_64-linux-gnu-g++' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for py3exiv2

Root cause: libboost fails to create correct symbolic links. We need to create them manually:

bafu@de-lab-cafe:/usr/lib/x86_64-linux-gnu$ ls -l | grep libboost_python
-rw-r--r--  1 root root    259776  四   1  2020 libboost_python38.so.1.71.0

bafu@de-lab-cafe:/usr/lib/x86_64-linux-gnu$ sudo ln -s libboost_python38.so.1.71.0 libboost_python38.so
bafu@de-lab-cafe:/usr/lib/x86_64-linux-gnu$ sudo ln -s libboost_python38.so libboost_python3.so

bafu@de-lab-cafe:/usr/lib/x86_64-linux-gnu$ ls -l | grep libboost_python
lrwxrwxrwx  1 root root        27  四  28 13:27 libboost_python38.so -> libboost_python38.so.1.71.0
-rw-r--r--  1 root root    259776  四   1  2020 libboost_python38.so.1.71.0
lrwxrwxrwx  1 root root        20  四  28 13:28 libboost_python3.so -> libboost_python38.so

then I can create correct CAI images on DLC server.

(venv) bafu@de-lab-cafe:~/debug$ sha256sum bread-cai-cai-cai.jpeg 
af30fc972c6538fb3fc14f940cf027df588345c8456c530f2f14ca85d2e6544e  bread-cai-cai-cai.jpeg

System dependencies on storage-backend dev, DLC server, and Bofu's devenv:

storage-backend dev

ubuntu@dia-dev:~$ apt-cache policy libexiv2-dev libboost1.71-dev swig
libexiv2-dev:
  Installed: 0.27.2-8ubuntu2
  Candidate: 0.27.2-8ubuntu2
  Version table:
 *** 0.27.2-8ubuntu2 500
        500 http://ap-southeast-1.ec2.archive.ubuntu.com/ubuntu focal/main amd64 Packages
        100 /var/lib/dpkg/status
libboost1.71-dev:
  Installed: 1.71.0-6ubuntu6
  Candidate: 1.71.0-6ubuntu6
  Version table:
 *** 1.71.0-6ubuntu6 500
        500 http://ap-southeast-1.ec2.archive.ubuntu.com/ubuntu focal/main amd64 Packages
        100 /var/lib/dpkg/status
swig:
  Installed: 4.0.1-5build1
  Candidate: 4.0.1-5build1
  Version table:
 *** 4.0.1-5build1 500
        500 http://ap-southeast-1.ec2.archive.ubuntu.com/ubuntu focal/universe amd64 Packages
        100 /var/lib/dpkg/status

DLC

bafu@de-lab-cafe:~/debug$ apt-cache policy libexiv2-dev libboost1.71-dev swig
libexiv2-dev:
  Installed: 0.27.2-8ubuntu2
  Candidate: 0.27.2-8ubuntu2
  Version table:
 *** 0.27.2-8ubuntu2 500
        500 http://tw.archive.ubuntu.com/ubuntu focal/main amd64 Packages
        100 /var/lib/dpkg/status
libboost1.71-dev:
  Installed: 1.71.0-6ubuntu6
  Candidate: 1.71.0-6ubuntu6
  Version table:
 *** 1.71.0-6ubuntu6 500
        500 http://tw.archive.ubuntu.com/ubuntu focal/main amd64 Packages
        100 /var/lib/dpkg/status
swig:
  Installed: 4.0.1-5build1
  Candidate: 4.0.1-5build1
  Version table:
 *** 4.0.1-5build1 500
        500 http://tw.archive.ubuntu.com/ubuntu focal/universe amd64 Packages
        500 http://tw.archive.ubuntu.com/ubuntu focal/universe i386 Packages
        100 /var/lib/dpkg/status

Bofu's devenv

apt-cache policy libexiv2-dev libboost1.71-dev swig
libexiv2-dev:
  Installed: 0.27.2-8ubuntu2
  Candidate: 0.27.2-8ubuntu2
  Version table:
 *** 0.27.2-8ubuntu2 500
        500 http://tw.archive.ubuntu.com/ubuntu focal/main amd64 Packages
        100 /var/lib/dpkg/status
libboost1.71-dev:
  Installed: 1.71.0-6ubuntu6
  Candidate: 1.71.0-6ubuntu6
  Version table:
 *** 1.71.0-6ubuntu6 500
        500 http://tw.archive.ubuntu.com/ubuntu focal/main amd64 Packages
        100 /var/lib/dpkg/status
swig:
  Installed: 4.0.1-5build1
  Candidate: 4.0.1-5build1
  Version table:
 *** 4.0.1-5build1 500
        500 http://tw.archive.ubuntu.com/ubuntu focal/universe amd64 Packages
        500 http://tw.archive.ubuntu.com/ubuntu focal/universe i386 Packages
        100 /var/lib/dpkg/status
shc261392 commented 3 years ago

Found another case of corrupted image from production site.

Here are 4 related images:

downtown-koto-ku.jpg (original image, has CAI info): https://drive.google.com/file/d/1Yz-jPigYHW5AlqRD24ChC8qfzlVnVHvC/view?usp=sharing

downtown-koto-ku-thumbnail.jpg (original thumbnail, has no CAI): https://drive.google.com/file/d/1xIQnrPNvABdcQi7jAX8j8FOJORWKxbLt/view?usp=sharing

downtown-koto-ku_cai3.jpg (CAI injected by backend, corrupted): https://drive.google.com/file/d/1ermu5KnqErZkWgLLnIsFVim2rA8Pm_fN/view?usp=sharing

downtown-koto-ku-cai-cai-cai.jpg (CAI injected with utils/starling_multiple_injection.py, corrupted): https://drive.google.com/file/d/1KPM82_O38sR2M3Eavp2vfhqSOmmB6-IM/view?usp=sharing

sha256sum

2b293fb20b9d36b46ce92d06d65be1d5bc8bf2e5d5f476ea68e0875d9b75b535 downtown-koto-ku-cai-cai-cai.jpg
8fde780878a3754759e014637b03a68e2434374e3ae56f4d8f7b255f36f2acfd downtown-koto-ku-thumbnail.jpg
4039757f72e4cc6152f282ab8820ebcdf44b4dc74ae56fe5c11cef7981d8ea46 downtown-koto-ku.jpg
c2ec4c752f64e7f78555b53d4cf6c2222a06b5d4204a4094af8c843598716881 downtown-koto-ku_cai__3__.jpg

Injecting the original image with the original thumbnail results in a corrupted image with correct CAI info. (can be extracted by codestream-parser/utils/cai-parser.py or verify website)

shc261392 commented 3 years ago

Another corrupted image: https://drive.google.com/file/d/1U8XhsSFdrSFLLrio1AuLA4wXJvHoQ95k/view?usp=sharing

It might be easier to find the the common trait of these images and find the root cause if we could find more cases like this

bafu commented 3 years ago

Image Viewer shows the error when opening downtown-koto-ku-cai-cai-cai.jpg

Error interpreting JPEG image file (Invalid JPEG file structure: SOS before SOF)

metadata structure in JPEG Syntax and structure

bafu commented 3 years ago

Question: Is thumbnail put at the right position?

Starting and ending bytes in downtown-koto-ku-thumbnail.jpg

0000:0000 | FF D8 FF E0  00 10 4A 46  49 46 00 01  01 00 00 01 | ÿØÿà..JFIF......
...
0000:3770 | D3 34 BB 89  5F A5 20 14  91 D8 62 80  38 E9 4C 1F | Ó4»._¥ ..Øb.8éL.
0000:3780 | 78 D0 72 3A  12 28 19 FF  D9                       | xÐr:.(.ÿÙ

CAI Injection position

0000:6D60 | 6B 65 74 20  65 6E 64 3D  22 77 22 3F  3E FF EB FF | ket end="w"?>ÿëÿ
0000:6D70 | FF 4A 50 00  01 00 00 00  01 00 01 EA  1E 6A 75 6D | ÿJP........ê.jum

Thumbnail position

0000:6F80 | 67 2E 6A 70  67 00 00 00  37 91 6A 70  32 63 FF D8 | g.jpg...7.jp2cÿØ
0000:6F90 | FF E0 00 10  4A 46 49 46  00 01 01 00  00 01 00 01 | ÿà..JFIF........
...
0000:A700 | BB 89 5F A5  20 14 91 D8  62 80 38 E9  4C 1F 78 D0 | »._¥ ..Øb.8éL.xÐ
0000:A710 | 72 3A 12 28  19 FF D9 00  00 37 D4 6A  75 6D 62 00 | r:.(.ÿÙ..7Ôjumb.

SOI & DQT positions (0xFFD8, 0xFFDB)

0000:03D0 | 00 00 00 01  FF D8 FF DB  00 84 00 02  01 02 02 02 | ....ÿØÿÛ........
0000:03E0 | 01 02 02 02  02 02 02 02  02 03 05 03  03 03 03 03 | ................

SOS position (0xFFDA)

0000:0610 | F8 F9 FA FF  DA 00 0C 03  01 00 02 11  03 11 00 3F | øùúÿÚ..........?
0000:0620 | 00 FD 1E FD  89 3E 3F 68  9A 6E 81 F0  9F E1 54 DE | .ý.ý.>?h.n.ð.áTÞ
bafu commented 3 years ago

downtown-koto-ku.jpg contains a CAI injection

$ exiv2 -p a pr downtown-koto-ku.jpg | grep Xmp
Xmp.dcterms.provenance                       XmpText    39  self#jumbf=cai/cb.Authmedia_1/cai.claim

CAI parsing result: downtown-koto-ku.txt

bafu commented 3 years ago

downtown-koto-ku-cai-cai-cai.jpg contains 4 CAI injections

$ exiv2 -p a pr downtown-koto-ku-cai-cai-cai.jpg | grep Xmp
Xmp.dcterms.provenance                       XmpText    49  self#jumbf=cai/cb.ThunderCoreNFTChain_3/cai.claim

CAI parsing result: downtown-koto-ku-cai-cai-cai.txt

There is a duplicate label cb.Authmedia_1

$ grep "LABEL: cb." downtown-koto-ku-cai-cai-cai.txt 
          LABEL: cb.Authmedia_1
          LABEL: cb.Authmedia_1
          LABEL: cb.IOTAIntegrityChain_2
          LABEL: cb.ThunderCoreNFTChain_3

However, it seems not be the root cause.

I did some experiments

  1. renamed one of the cb.Authmedia_1 to cb.Authmedia_2
  2. removed cb.Authmedia_1 in the multi-injection script

the output images are still broken.

bafu commented 3 years ago

It seems that "single injection + 3 multiple injection" breaks the DQT marker data.

downtown-koto-ku.jpg, single injection

Screenshot from 2021-06-15 19-06-47

downtown-koto-ku-cai-cai-cai.jpg, single injection + 3 multiple injections

Screenshot from 2021-06-15 19-09-36

bafu commented 3 years ago

James 6:09 PM @bafu 我在資料夾放了兩張原本 inject 會 corrupted 的 image https://drive.google.com/drive/folders/1V_-aT9QPOtGggixasIGhcF5Je0W3nIdn?usp=sharing 6:10 labrador 跟 sE7

scott-dt commented 3 years ago

@bafu 可以幫我是這兩張嗎? 先前說不能 inject 的圖 20201024_150332 20210214_114838

bafu commented 3 years ago

Fixed by commit 1dcf02a.

bafu commented 3 years ago

DQT #00 disappears in the injected photo.

$ sha256sum tt-2447-*
4c4586b2e75a96d8ad819501c69752105070d2922bc633ee79895cc052c7c374  tt-2447-broken.jpg
efea73976ab0c5ead97aeb3477bcbd75f4b9c5ff8dba12527235342425968b2a  tt-2447-original.jpg