joz-k / ios_backup_extractor

iOS backup media extraction tool (photos, videos)
Other
30 stars 1 forks source link

[request] preserve media creation and modification date and time #4

Closed Box333 closed 1 week ago

Box333 commented 3 weeks ago

Hi,

thank you for the good work.

would be nice, if you could add a switch to preserve media creation and modification date and times.

kind regards

joz-k commented 3 weeks ago

Hi. I am happy to help, but I need to know more about this request. The Exif data is preserved for each photo (e.g. "Date Taken", "F Number", "Exposure Time", "GPS", and all other info).

So is this about having this as part of a filename?

Box333 commented 2 weeks ago

its about keeping the the original creation and modifications dates (=filesystem metadata, not exif) of images when they are extracted to pc.

these dates are stored in the bplist $bplist_obj->{'$objects'}[1]{Birth} $bplist_obj->{'$objects'}[1]{LastModified}

for example, if you connect an iphone to pc, you will see in windows explorer: \:Apple iPhone\Internal Storage\DCIM\101APPLE\IMG_0100.jpg | Created: 2022-11-01 10:00 | LastModified: 2022-11-02 10:00

as you know, these dates are stored in the manifest.db files table in the bplist blob Birth = 2022-11-01 10:00 LastModified = 2022-11-02 10:00

so, after IMG_0100.jpg is extracted from backup, you have to apply the both dates from the bplist to the extracted JPG.

There is a pyhton exaple for this (look for ctime and mtime): 'https://github.com/jsharkey13/iphone_backup_decrypt/blob/master/src/iphone_backup_decrypt/utils.py' , see last line

Box333 commented 2 weeks ago

btw, do you plan to make a version with encrypted backup support ?

joz-k commented 2 weeks ago

OK, thank you. I think I understand what you're asking now. You want the file creation and modification time (filesystem) attributes for the media files to be the same as stored in the manifest. I will look into this.

BTW, the linked Python script changes only one of the attributes. It changes "Access" and "Modified" time of the media file to the "LastModified" time from the manifest. It doesn't change the "Created" time attribute to mirror the "Birth" date. And Unix-based operating systems don't track the "DateCreated" attribute, although MacOS has an extension for it.

Do you need this primarily for Windows or MacOS/Unix? The solution may differ depending on the OS, so I'm asking for priority.

Box333 commented 2 weeks ago

yes, you understood everything 100%! i need it for windows

I think its essential when you extract 1000s of images to have the original creation dates.

To get the ctime, i installed a package called "win32_setctime" and modified the utily.py.

pip install win32_setctime

utily.py: from win32_setctime import setctime

def _init_
     self.ctime = self.data.get("Birth")

and at the end:

if file_plist.ctime:
        setctime(out_filepath, file_plist.ctime)
joz-k commented 2 weeks ago

All right. Although I don't know your particular use case, I hope you also noticed the --format parameter, which can sort your photos down to the day level.

I wonder if adding the option to export media files with modified filenames like:

2024-11-03_IMG_6626.heic

instead of the original IMG_6626.heic, where 2024-11-03 is the "Birth" timestamp, would also serve your needs just as well. Personally, I do not trust file system attributes too much.. When you move files around, they can change because some tool/filesystem does not preserve them.

Box333 commented 2 weeks ago

Hi, of course i have noticed and also alreday used the --format parameter.

Of course you could additionally add another parameter for prepending modified date or other information to filenames.

the important part though is, that the files are extracted with the original filesystem dates.

,,,and if your script would support enctrypted backups, this would be perfect. you can find everything you need in (https://github.com/jsharkey13/iphone_backup_decrypt). maybe traslate to perl with AI. ?

rb0022 commented 2 weeks ago

@joz-k Firstly, fantastic work on an annoying problem.

I agree preservation of the metadata is necessary as the OP mentions. In a brief test extracting images via Photo.app and looking in Package Contents/originals/0/file.jpg versus the same file with your extractor, just looking in macOS Get Info without diving deeper into the attribs....

The following is altered by the extractor and preserved by macOS Photo.app:

While the subdirectory structure in YYYY-MM or YYYY-MM-DD is excellent at organising the files themselves, the lack of date metadata will result in confusion in most gallery-style applications and other use cases. As the OP described, it is not good to have thousands of JPGs not with the original datetime attribs.

N.B. your suggestion to optionally append to the file (as you suggested 2024-11-03_IMG_6626.heic) would be a very nice too. However, I would strongly suggest providing an 3 optional formats, as you suggest with hyphens, use underscore (i.e. 2024_11_03_IMG_6626.heic) or none (i.e. 20241103_IMG_6626.heic).


@Box333 please adhere to GH etiquette and raise your encrypted backups request, in a separate GH Issue

joz-k commented 2 weeks ago

The following is altered by the extractor and preserved by macOS Photo.app:

  • Created Date (datetime)
  • Modified Date (datetime)
  • Latitude
  • Longitude

@rb0022 I agree with you. A minor technical detail: these attributes are not preserved, they are recreated by Photo.app. And this tool should do it as well.

joz-k commented 1 week ago

I have implemented the fix to restore the DataModified/DateCreated filesystem attributes for each exported file.

I have also added the --prepend-date and --prepend-date-separator options to add a date timestamp to each exported filename. Supported formats are:

After the next binary release, I will close this issue.

rb0022 commented 1 week ago

@joz-k Looks good. Were there any limitations the fallback utime system call, as opposed to the method with SetFile (Xcode Tools) ? It is not clear in the commented code if there is a limitation

Separate GH Issues to track the following I assume?

joz-k commented 1 week ago

Looks good. Were there any limitations the fallback utime system call, as opposed to the method with SetFile (Xcode Tools) ? It is not clear in the commented code if there is a limitation

The limitation of using utime on MacOS is the fact that utime is a POSIX (Unix) system call and the plain Unix filesystem has no notion of a "DateCreated" file system attribute. Therefore, utime sets the "modification time" and the "access time" (which is usually disabled for performance reasons and only reflects the "modification time"). On MacOS (probably on Linux too, but I haven't tested it), if the "DateModified" is changed by the utime call and is older than the "DateCreated", the system changes the "DateCreated" to be the same as the "DateModified". For this utility, this condition is clearly always met because the photo/video is never newer than the date of extraction from the backup. This means that "DateCreated" is always set to the "DateModified" attribute. I have found very few photos in my library where "DateCreated" and "DateModified" are off by more than a few seconds. Probably when the user applies the filter (or edits the photo in some other way) these two dates can differ. In practice this is rare and not very significant.

Apparently, the correct solution would be to use either the CFURLSetResourcePropertyForKey C function from the Core Foundation or an Objective-C NSFileManager from the Cocoa API. Neither is accessible from any Perl module I could find.

Separate GH Issues to track the following I assume?

  • Encrypted backup
  • Latitude / Longitude attribs

Yes, please.

joz-k commented 1 week ago

The main point of this request has been implemented in the 1.2.4 release.

rb0022 commented 1 week ago

@joz-k Thank you for the hard work. I have raised the 2x additional requests in separate GH Issues.

Personally, I would suggest the functionality of the additional file attribs is higher value than the encrypted backup compat; on the assumption that a user should just perform an unencrypted backup / use another tool to decrypt before using ios_backup_extractor.