During export, Ignore edits that are only rotations?

[felciano]

Is your feature request related to a problem? Please describe. I've started using osxphotos to export images from a handful of libraries. In many cases I end up with "duplicated" photos exported in the sense that a image.jpg is exported as well as an image_edited.jpg alternative. In many cases, the only "edit" in the second one is that the photo was rotated, which is frustrating because it doubles the amount of exported files for essentially a trivial change.

Describe the solution you'd like I would love a --ignore-rotations flag that will not export edited photos if their only edit is a rotation.

Describe alternatives you've considered I've tried opening the exported photos in Preview and rotating to the same orientation, but unfortunately this results in different file sizes. So something about the rotate operation and/or the subsequent export is lossy, such that an subsequent rotation won't undo the change.

Additional context I would be OK with a significant delay in the export process to support this operation. For example, if there is a native Photos way to un-rotate and then compare the images to confirm they are now identical, that might do the trick.

[RhetTbull]

This might be possible by examine the adjustments data decoded by osxphotos.

Rotated image, no edits:

No adjustments and orientation is set.

 {
    "editor": "com.apple.Photos",
    "format_id": "com.apple.photo",
    "base_version": 0,
    "format_version": "1.4",
    "adjustments": [],
    "metadata": {
      "masterHeight": 3024,
      "masterWidth": 3024,
      "orientation": 8
    },
    "orientation": 8,
    "adjustment_format_version": 1,
    "version_info": {
      "buildNumber": "22G90",
      "appVersion": "560.0.110",
      "schemaRevision": 0,
      "platform": "macOS"
    },
    "timestamp": "2024-06-21T05:30:21+00:00"
  }

Edited image:

"adjustments" contains data (but also orientation is set even though image not rotated)

{
    "editor": "com.apple.Photos",
    "format_id": "com.apple.photo",
    "base_version": 0,
    "format_version": "1.4",
    "adjustments": [
      {
        "formatVersion": 1,
        "enabled": 1,
        "settings": {
          "offsetTone": 0,
          "offsetGrain": 0,
          "offsetStrength": 0,
          "offsetNeutralGamma": 0,
          "auto": 0,
          "inputBlackAndWhite": 0.04888947315705128
        },
        "identifier": "SmartBlackAndWhite",
        "formatIdentifier": "com.apple.photo"
      }
    ],
    "metadata": {
      "masterHeight": 3024,
      "masterWidth": 3024,
      "orientation": 1
    },
    "orientation": 1,
    "adjustment_format_version": 1,
    "version_info": {
      "buildNumber": "22G90",
      "appVersion": "560.0.110",
      "schemaRevision": 0,
      "platform": "macOS"
    },
    "timestamp": "2024-06-21T05:34:03+00:00"
  }

Edited and rotated image

"adjustments" contains data and orientation is also set.

{
    "editor": "com.apple.Photos",
    "format_id": "com.apple.photo",
    "base_version": 0,
    "format_version": "1.4",
    "adjustments": [
      {
        "formatVersion": 1,
        "enabled": 1,
        "settings": {
          "offsetTone": 0,
          "offsetGrain": 0,
          "offsetStrength": 0,
          "offsetNeutralGamma": 0,
          "auto": 0,
          "inputBlackAndWhite": 0.23603640825320515
        },
        "identifier": "SmartBlackAndWhite",
        "formatIdentifier": "com.apple.photo"
      }
    ],
    "metadata": {
      "masterHeight": 3024,
      "masterWidth": 3024,
      "orientation": 3
    },
    "orientation": 3,
    "adjustment_format_version": 1,
    "version_info": {
      "buildNumber": "22G90",
      "appVersion": "560.0.110",
      "schemaRevision": 0,
      "platform": "macOS"
    },
    "timestamp": "2024-06-21T05:38:02+00:00"
  }

It appears that if the photo shows as edited and "adjustments" is null, then the photo has been rotated but not edited. This could be used to filter the images that were only rotated.

[RhetTbull]

I wonder if a more general purpose option would be better?

For example, --skip-edited-if TEMPLATE would evaluate a template, in much the way --query-eval TEMPLATE does.

To skip rotated images then, you'd do:

--skip-edited-if "photo.adjustments and not photo.adjustments.adjustments"

This would skip any edited version if there was an adjustments file but the adjustments data was missing. The and clause is required to prevent a KeyError for trying to access adjustments data for a photo that doesn't have adjustments.

It's not as intuitive perhaps as --ignore-rotations but is much more versatile and can be adapted for other use cases.

[felciano]

Wow--I love that idea. Way more extensible than what I proposed

[felciano]

I'm just learning osxphotos, but if I want to test your hypothesis, can I generate the above sample JSON output myself and then use that to compare two images?

[RhetTbull]

@felciano If you save the following as "adjustments.py" then select some photos in Photos and run the command with osxphotos run adjustments.py, it will dump the JSON. You can't easily directly dump it as you need to drop the data field (contains the raw binary for the adjustment data which doesn't work with JSON) so the script drops this and adds some additional data about the photo.

"""Print out JSON for adjustments info for selected photos

Run with: osxphotos run adjustments.py
"""

from __future__ import annotations

import json

import osxphotos
from osxphotos.cli import selection_command

@selection_command
def adjustments(photos: list[osxphotos.PhotoInfo], **kwargs):
    """Prints out the adjustments info, if any, for each selected photo as JSON."""

    json_data = []
    for photo in photos:
        data = {
            "uuid": photo.uuid,
            "filename": photo.original_filename,
            "hasadjustments": photo.hasadjustments,
        }
        if photo.adjustments:
            data |= photo.adjustments.asdict()
            data.pop("data")  # remove the raw data field
        json_data.append(data)
    print(json.dumps(json_data, indent=4))

if __name__ == "__main__":
    adjustments()

[RhetTbull]

We could do the same for originals: --skip-original-if TEMPLATE and perhaps RAW for RAW+JPEG: --skip-raw-if TEMPLATE

[oPromessa]

Would this example help? export.py

Alternatively, @RhetTbull I recall you've made available in the API all the export options/functionality so that one could take the benefit of all the options plus some added control via API. Don't seem to find that reference.

[RhetTbull]

@oPromessa I don't think it's in the docs (on mobile so not easy to check) but export_cli() in osxphotos/cli/export.py can be called with any export options you want to have programmatic access to the export command.

https://github.com/RhetTbull/osxphotos/blob/5191041b4f58e04c0cd4ea8b4ae1e50a7e6bc8fc/osxphotos/cli/export.py#L1119

This won't solve the problem though because it lacks logic to skip only certain edits. I think this can be implemented fairly easily though and once done, you'll be able to use any criteria you want for determining which assets are exported.

[RhetTbull]

I looked at the code needed to implement this. I think this will be fairly straightforward to implement. The ExportOptions class used to pass options to the exporter already has flags for original, live, preview, edited, and raw assets. There's already code that evaluates the criteria for --query-eval that can be re-used to evaluate the criteria for these new options so putting it all together just means adding the options, wrapping existing code, and writing tests.

[felciano]

@felciano If you save the following as "adjustments.py" then select some photos in Photos and run the command with osxphotos run adjustments.py, it will dump the JSON. You can't easily directly dump it as you need to drop the data field (contains the raw binary for the adjustment data which doesn't work with JSON) so the script drops this and adds some additional data about the photo.

I confirmed this dumps info about the currently selected items, and any untouched photo lacks the adjustments value. Note that some of those values differ depending on the original orientation of the photo (e.g. landscape to portrait, or vice versa). But other than that, seems like this should work for detecting rotated photos.

Is there an obvious way to implement logic along the lines of "ignore rotated photos if the unrotated version has already been exported"? This would support the original use case (above) but also work for cases where you make a copy to correct a photo rotation and then delete the original (although that is rare in my case)

[RhetTbull]

@all-contributors please add @felciano for research, ideas

[RhetTbull]

Is there an obvious way to implement logic along the lines of "ignore rotated photos if the unrotated version has already been exported"? ... but also work for cases where you make a copy to correct a photo rotation and then delete the original

I don't think this would be easy outside the original use case (once I implement the --skip-edited-if option). If you make a copy of a photo, osxphotos no longer recognizes this as associated with the original photo. Photos assigns a unique ID (UUID) to each photo and OSXPhotos uses this to track / associate exported assets and the assets in Photos. A duplicate would have a different UUID. OSXPhotos does contain logic to find duplicates (used for osxphotos import) by computing a unique fingerprint for each photo (same algorithm that Photos uses). This could theoretically used for export for a --skip-dups option (not something I've really thought about) but the added logic of "but only skip if non-rotated version is already exported" would be complex to implement for what is arguably a very niche edge case.

[allcontributors[bot]]

@RhetTbull

We had trouble processing your request. Please try again later.

[felciano]

@RhetTbull regarding the "ignore rotated photos if the unrotated version has already been exported" explanation: thanks for clarifying; makes sense that would be hard to implement in osxphotos as is.

Let me provide a broader use case for this feature request:

• Say I have a folder of images on my drive.
• I think they've already been imported to the Photos app, but I'm not sure.
• So I want to run some sort of comparison between the two.
• A simple approach would be to export the photos from Photos app using osxphotos (maybe constrained by date range based on the earliest/latest photos in the folder), and then run some sort of diff.
• Currently, this process will miss pseudo-duplicates, where the exported image is technically different in that it was rotated in the Photos app.

Is there perhaps a different way to think about using osxphotos to address this "are these photos already in the Photos app, including perhaps rotated" task?

[RhetTbull]

@felciano because edits (including rotations) in Photos are always non-destructive, the original image is preserved. This means you can check an exported original against originals in Photos. A rotation will only affect the edited version that is exported or the original if exported with --exiftool. In fact, OSXPhotos includes a tool for doing this type of check: osxphotos import --check.

For example, I exported a single image using osxphotos export:

osxphotos export ~/Desktop/export --verbose --selected
osxphotos version: 0.68.2
...
Done processing details from Photos library.
Exporting 1 photo to /Users/rhet/Desktop/export...
Exporting wedding.jpg (E9BC5C36-7CD1-40A1-A72B-8B8FAC227D51.jpeg) (1/1)
Exported wedding.jpg to /Users/rhet/Desktop/export/wedding.jpg
Exported /Users/rhet/Desktop/export/wedding.jpg
Exporting 1 photos ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
Processed: 1 photo, exported: 1, missing: 0, error: 0
Elapsed time: 0:00:00
Cleaning up lock files

Then I rotated the image in Photos and ran osxphotos import --check to verify the exported image matched:

❯ osxphotos import ~/Desktop/export --check
Collecting files to import... 
Filtering import list for image & video files... 
Grouping files by parent directory... 
Grouping files into import groups...
✅️ /Users/rhet/Desktop/export/wedding.jpg, imported, wedding.jpg (E9BC5C36-7CD1-40A1-A72B-8B8FAC227D51) added 2019-07-27 09:16:49.735651-04:00

OSXPhotos correctly detects that the exported original matches an asset already in the library. Now, if you applied the rotation by changing the EXIF orientation flag then the photos would not match -- the import check only verifies exact duplicates. To do so, it uses a fingerprint (file hash) function that is also used by Photos and conveniently stored in the Photos database. Thus OSXPhotos does not need to compute the hash for every photo in the library, it can simply query the database to see if there's a match. If you wanted to do this for an image with EXIF data that changed, you'd have to use some sort of "visual diff". Such algorithms exist but this would be much more computationally complex. Perhaps an idea for a future osxphotos feature?

[oPromessa]

Use --skip-edit

• A simple approach would be to export the photos from Photos app using osxphotos (maybe constrained by date range based on the earliest/latest photos in the folder), and then run some sort of diff.

What if you use --skip-edit option to actually have osxphotos only export originals. You can then compare them with your folder. You can also use the --export-as-hardlink to save space on exported files by linking directly to the actual original file (does not work with --exiftool).

You may also use Gemini 2 to find duplicated images across your two folders.

• Gemini 2 can also delete one of the copies of the duplicate files you select. You should only delete the ones from your folder.
• the leftovers are photos non-existent on Photos.

Photos duplicates check on import

Alternatively, when you select a folder to import in Photos it does try to find duplicates already imported; prprior to actually import them. I don't fully trust it 😉 but if they are actually the very same it may give you a pointer to duplicates.

[felciano]

@RhetTbull thanks for the pointers. I've tried this workflow with Gemini 2, and can't figure out how to get it to reliably only elect to delete duplicates from one of the two folders. Despite indicating that it should never delete from under a particular root tree, when I tell it to "Select Any" file to delete, it will sometimes pick the duplicate that since under that "never touch" space. I've reproduced this and filed a report with the developers, but no reply.

I've ended hacking my around it by doing full exports, and then using https://github.com/pkolaczk/fclones with some post-processing to find the duplicates to delete.