Add --album-uuid to export command

[mkirkland4874]

Add a --album-uuid option to the export command. The --album option will combine all albums that have the same name (even if in different folders). Using the album UUID would allow us to export specific albums even if they have the same name as others without combining the albums.

[RhetTbull]

To make this useful, I should also add the uuid to the the osxphotos albums command so you can easily find the the right uuid. In the mean time, the following should work using --query-eval:

First, to get uuids of albums, enter the following commands into the osxphotos repl mode:

for album in photosdb.album_info:
    print(album.uuid, album.title, album.folder_names)

From the results of this command, you can find the uuid(s) for the albums you want (see output below) then you can use --query-eval to filter the right photos:

osxphotos export ~/Desktop/export -V --query-eval " '973ED0FD-5B5F-4CD7-A40F-4DDE73CE3FAB' in [a.uuid for a in photo.album_info]"

--query-eval works by running a python list comprehension on the list of all photos. The above --query-eval is equivalent to running this in the osxphotos repl mode:

[photo for photo in photos if '973ED0FD-5B5F-4CD7-A40F-4DDE73CE3FAB' in [a.uuid for a in photo.album_info]]

$ osxphotos repl
python version: 3.9.5 (v3.9.5:0a7dcbdb13, May  3 2021, 13:17:02)
[Clang 6.0 (clang-600.0.57)]
osxphotos version: 0.42.59
Using last opened Photos library: /Users/rhet/Pictures/Test-10.15.7.photoslibrary
Loading database
Processing database /Users/rhet/Pictures/Test-10.15.7.photoslibrary/database/photos.db
Processing database /Users/rhet/Pictures/Test-10.15.7.photoslibrary/database/Photos.sqlite
Database locked, creating temporary copy.
Processing database.
Database version: 6000, 5.
Processing persons in photos.
Processing detected faces in photos.
Processing albums.
Processing keywords.
Processing photo details.
Processing import sessions.
Processing additional photo details.
Processing face details.
Processing photo labels.
Processing EXIF details.
Processing computed aesthetic scores.
Processing comments and likes for shared photos.
Done processing details from Photos library.
Done: took 0.12 seconds
Getting photos
Found 25 photos in 0.01 seconds
The following variables are defined:
- photosdb: PhotosDB() instance for /Users/rhet/Pictures/Test-10.15.7.photoslibrary
- photos: list of PhotoInfo objects for all photos in photosdb, including those in the trash

The following functions may be helpful:
- get_photo(uuid): return a PhotoInfo object for photo with uuid
- get_selected(): return list of PhotoInfo objects for photos selected in Photos
- show(photo): open a photo object in the default viewer
- help(object): print help text including list of methods for object; for example, help(PhotosDB)
- quit(): exit this interactive shell

>>> for album in photosdb.album_info:
...     print(album.uuid, album.title, album.folder_names)
...
0C514A98-7B77-4E4F-801B-364B7B65EAFA Pumpkin Farm []
ECB9B3AA-7BEF-474D-90FA-A104EC42ED22 Test Album []
AA4145F5-098C-496E-9197-B7584958FF9B Test Album []
973ED0FD-5B5F-4CD7-A40F-4DDE73CE3FAB AlbumInFolder ['Folder1', 'SubFolder2']
68001ACE-DE4E-46B7-A3F8-3B2D03D39D59 Raw ['Folder2']
D4DC7467-1F13-46E8-86BC-540FB059463C EmptyAlbum []
EA8E27F6-2A49-44B0-BC77-2A2BC23C21BF I have a deleted twin []
8A3F182E-20D4-4C30-AFFD-9BD7CEC01C75 2018-10 - Sponsion, Museum, Frühstück, Römermuseum []
3ABA0FAD-470D-41D7-BDA9-C46D2662AC04 2019-10/11 Paris Clermont []
05CD2501-762A-475B-A9A6-055C6E90269C Multi Keyword []
>>>

[RhetTbull]

This will give slightly prettier output in the repl to show full path to each album and count of photos in the album:

>>> for album in photosdb.album_info:
...     print(album.uuid, '/'.join([*album.folder_names, album.title]), len(album.photos))
...
0C514A98-7B77-4E4F-801B-364B7B65EAFA Pumpkin Farm 3
ECB9B3AA-7BEF-474D-90FA-A104EC42ED22 Test Album 1
AA4145F5-098C-496E-9197-B7584958FF9B Test Album 1
973ED0FD-5B5F-4CD7-A40F-4DDE73CE3FAB Folder1/SubFolder2/AlbumInFolder 2
68001ACE-DE4E-46B7-A3F8-3B2D03D39D59 Folder2/Raw 4
D4DC7467-1F13-46E8-86BC-540FB059463C EmptyAlbum 0
EA8E27F6-2A49-44B0-BC77-2A2BC23C21BF I have a deleted twin 1
8A3F182E-20D4-4C30-AFFD-9BD7CEC01C75 2018-10 - Sponsion, Museum, Frühstück, Römermuseum 1
3ABA0FAD-470D-41D7-BDA9-C46D2662AC04 2019-10/11 Paris Clermont 1
05CD2501-762A-475B-A9A6-055C6E90269C Multi Keyword 3

[RhetTbull]

If you're familiar with python, you could also use the --query-function option to use a custom python function to do this. Here's an example that makes a simple interactive query builder that lets you select albums by UUID. Save the code below to album_picker.py then run run the export as follows:

osxphotos export /path/to/export --query-function album_picker.py::albums

This function was built using the query_function.py example in the /examples folder in the osxphotos GitGub repo.

""" example function for osxphotos --query-function """

from typing import List

from osxphotos import PhotoInfo

# call this with --query-function examples/query_function.py::albums
def albums(photos: List[PhotoInfo]) -> List[PhotoInfo]:
    """your query function should take a list of PhotoInfo objects and return a list of PhotoInfo objects (or empty list)"""

    # need to get access to the underlying PhotosDB object to get all albums
    # grab it from the first photo in the list (PhotoInfo._db is the PhotosDB instance)
    albums = photos[0]._db.album_info
    album_details = [
        (album.uuid, "/".join([*album.folder_names, album.title]), len(album.photos))
        for album in albums
    ]

    print("uuid path photos")
    for album in album_details:
        print(*album)

    uuids = input("Enter list of album uuids, separated by comma to export: ")
    uuid_list = uuids.split(",")
    uuid_list = [uuid.strip() for uuid in uuid_list]

    return [
        photo
        for photo in photos
        if any(uuid in uuid_list for uuid in [album.uuid for album in photo.album_info])
    ]

[mkirkland4874]

@RhetTbull thanks, this is very helpful! I am using the osxphotos python package to get the album info and folder structure via album_info to dynamically get the albums before running commands to export all of the albums in the same folder structure as in Photos. Essentially I'm writing a python script to export/sync an exact copy of Photos including the default folders (All Photos, Favorites, Hidden, Recently Deleted) and also the entire album structure, so the album UUID would be very helpful to use as part of the export command. I'm then putting the entire export into a git-annex repo so each photo is stored only once (i.e., the storage layer is deduplicated even if photos are duplicated across folders/albums) and to keep snapshots of the backups over time (i.e., each time I sync it with Photos using the --update --cleanup options, I commit the changes to the git-annex repo) . For example, my final output folder structure would be something like the following (which requires running a separate export command for each folder/album):

All Photos
Favorites
Hidden
Recently Deleted
Albums
-- Album 1         (<--- need to export via UUID as it's a duplicate album name)
-- Album 2
-- Album Folder 1
---- Album 1       (<--- need to export via UUID as it's a duplicate album name)
---- Album 3
-- Album Folder 2
---- Album 4

I agree adding UUID to the albums command would be helpful as well but as you pointed out there are workarounds.

[RhetTbull]

@mkirkland4874 I think you can accomplish this export using a function template. For example, this should get you pretty close. Save the code below to export_template.py then run osxphotos using this command:

osxphotos export /path/to/export --directory "{function:export_template.py::directory}" -V --deleted --update

You need to use --deleted to also include recently deleted items. \

You'll also likely get name collisions if you export all images to "All Photos" but osxphotos will take care of this by adding (1), (2) etc. as needed to the filename.

For example, my final output folder structure would be something like the following (which requires running a separate export command for each folder/album):

Using a template function let's you do this with a single osxphotos export command!

Update the folder/album logic was wrong in the previous example as it didn't work right on folders or albums with embedded path separators ('/') so I switched to using osxphotos' own {folder_album} template which handles these cases.

""" Example showing how to use a custom function for osxphotos {function} template 
    Use:  osxphotos export /path/to/export --filename "{function:/path/to/template_function.py::example}"

    You may place more than one template function in a single file as each is called by name using the {function:file.py::function_name} format
"""

from typing import List, Union

import osxphotos
from osxphotos.phototemplate import RenderOptions

def directory(photo: osxphotos.PhotoInfo, **kwargs) -> Union[List, str]:
    """example function for {function} template; creates export directory to match Photos structure

    Args:
        photo: osxphotos.PhotoInfo object
        **kwargs: not currently used, placeholder to keep functions compatible with possible changes to {function}

    Returns:
        str or list of str of values that should be substituted for the {function} template
    """

    directories = ["All Photos"]
    if photo.favorite:
        directories.append("Favorites")
    if photo.hidden:
        directories.append("Hidden")
    if photo.intrash:
        directories.append("Recently Deleted")

    # render the folders and albums in folder/subfolder/album format
    # the __NO_ALBUM__ is used as a sentinel to strip out photos not in an album
    folder_albums, _ = photo.render_template(
        "{folder_album}", RenderOptions(dirname=True, none_str="__NO_ALBUM__")
    )
    if folder_albums:
        directories.extend(
            [
                "Albums/" + folder_album
                for folder_album in folder_albums
                if folder_album != "__NO_ALBUM__"
            ]
        )

    return directories

[RhetTbull]

The template function above produces the following export against one of the osxphotos test libraries which I think is pretty close to what you were looking for:

.
├── Albums
│   ├── 2018-10\ -\ Sponsion,\ Museum,\ Frühstück,\ Römermuseum
│   │   └── IMG_4547.jpg
│   ├── 2019-10:11\ Paris\ Clermont
│   │   └── IMG_4547.jpg
│   ├── Folder1
│   │   └── SubFolder2
│   │       └── AlbumInFolder
│   │           ├── IMG_4547.jpg
│   │           └── wedding_edited.jpeg
│   ├── Folder2
│   │   └── Raw
│   │       ├── DSC03584.dng
│   │       ├── IMG_1994.JPG
│   │       ├── IMG_1994.cr2
│   │       ├── IMG_1994_edited.jpeg
│   │       ├── IMG_1997.JPG
│   │       └── IMG_1997.cr2
│   ├── I\ have\ a\ deleted\ twin
│   │   └── wedding_edited.jpeg
│   ├── Multi\ Keyword
│   │   ├── Pumkins2.jpg
│   │   └── wedding_edited.jpeg
│   ├── Pumpkin\ Farm
│   │   ├── Pumkins1.jpg
│   │   ├── Pumkins2.jpg
│   │   └── Pumpkins3.jpg
│   └── Test\ Album
│       ├── Pumkins1.jpg
│       └── Pumkins2.jpg
├── All\ Photos
│   ├── DSC03584.dng
│   ├── IMG_1064.jpeg
│   ├── IMG_1693.tif
│   ├── IMG_1994\ (1).jpeg
│   ├── IMG_1994.JPG
│   ├── IMG_1994.cr2
│   ├── IMG_1994.jpeg
│   ├── IMG_1994_edited.jpeg
│   ├── IMG_1997.JPG
│   ├── IMG_1997.cr2
│   ├── IMG_3092.heic
│   ├── IMG_3092_edited.jpeg
│   ├── IMG_4547.jpg
│   ├── IMG_4957.JPG
│   ├── Jellyfish.MOV
│   ├── Jellyfish1.mp4
│   ├── Pumkins1.jpg
│   ├── Pumkins2.jpg
│   ├── Pumpkins3.jpg
│   ├── St\ James\ Park.jpg
│   ├── St\ James\ Park_edited.jpeg
│   ├── Tulips.jpg
│   ├── Tulips_edited.jpeg
│   ├── screenshot-really-a-png.jpeg
│   ├── wedding_edited.jpeg
│   ├── wedding_edited.jpg
│   ├── winebottle\ (1).jpeg
│   └── winebottle.jpeg
├── Favorites
│   └── wedding_edited.jpeg
├── Hidden
│   └── Pumpkins3.jpg
├── Recently\ Deleted
│   ├── IMG_1064.jpeg
│   ├── IMG_1994\ (1).jpeg
│   ├── IMG_1994.cr2
│   ├── IMG_1994.jpeg
│   └── wedding_edited.jpg
└── tree.txt

16 directories, 54 files

[mkirkland4874]

@RhetTbull This works great! I did not fully appreciate the templating and export function capabilities. What would be the best way to update the "All Photos" to export as "All Photos/{created.year}/{created.mm}/{created.dd}"?

[RhetTbull]

What would be the best way to update the "All Photos" to export as "All Photos/{created.year}/{created.mm}/{created.dd}"?

@mkirkland4874 I would just use the built in rendering engine which is accessible through PhotoInfo.render_template, just like for folder_album:

""" Example showing how to use a custom function for osxphotos {function} template 
    Use:  osxphotos export /path/to/export --filename "{function:/path/to/template_function.py::example}"

    You may place more than one template function in a single file as each is called by name using the {function:file.py::function_name} format
"""

from typing import List, Union

import osxphotos
from osxphotos.phototemplate import RenderOptions

def directory(photo: osxphotos.PhotoInfo, **kwargs) -> Union[List, str]:
    """example function for {function} template; creates export directory to match Photos structure

    Args:
        photo: osxphotos.PhotoInfo object
        **kwargs: not currently used, placeholder to keep functions compatible with possible changes to {function}

    Returns:
        str or list of str of values that should be substituted for the {function} template
    """

    # set directories to [All Photos/year/mm/dd]
    # render_template returns a tuple of [rendered value(s)], [unmatched]
    # here, we can ignore the unmatched value, assigned to _, as we know template will match
    directories, _ = photo.render_template("All Photos/{created.year}/{created.mm}/{created.dd}")
    if photo.favorite:
        directories.append("Favorites")
    if photo.hidden:
        directories.append("Hidden")
    if photo.intrash:
        directories.append("Recently Deleted")

    # render the folders and albums in folder/subfolder/album format
    # the __NO_ALBUM__ is used as a sentinel to strip out photos not in an album
    # use RenderOptions.dirname to force the rendered folder_album value to be sanitized as a valid path
    # use RenderOptions.none_str to specify custom value for any photo that doesn't belong to an album so
    # those can be filtered out; if not specified, none_str is "_"
    folder_albums, _ = photo.render_template(
        "{folder_album}", RenderOptions(dirname=True, none_str="__NO_ALBUM__")
    )
    if folder_albums:
        directories.extend(
            [
                "Albums/" + folder_album
                for folder_album in folder_albums
                if folder_album != "__NO_ALBUM__"
            ]
        )

    return directories

[RhetTbull]

I'm then putting the entire export into a git-annex repo so each photo is stored only once (i.e., the storage layer is deduplicated even if photos are duplicated across folders/albums) and to keep snapshots of the backups over time (i.e., each time I sync it with Photos using the --update --cleanup options, I commit the changes to the git-annex repo)

I wasn't familiar with git-annex but I love this idea! Having a git tracked copy of the Photos library is a great idea.

[mkirkland4874]

@mkirkland4874 I would just use the built in rendering engine which is accessible through PhotoInfo.render_template, just like for folder_album:

Thanks, this works perfectly!

I wasn't familiar with git-annex but I love this idea! Having a git tracked copy of the Photos library is a great idea.

It's great as it only stores 1 copy of the same file (no additional storage costs to exporting the way I am doing above) and you can use branches/tags to keep snapshots of the library at points in time. If no branches/tags use a file that is in the repo, you can use dropunused to remove that file from the repo (for example, if you want to purge deleted files after 1 year you drop any branches/tags for snapshots older than a year and run dropunused).

[mkirkland4874]

Here is a slightly updated version to mimic the presentation of photos in Photos.app as closely as possible.

""" Example showing how to use a custom function for osxphotos {function} template 
    Use:  osxphotos export /path/to/export --filename "{function:/path/to/template_function.py::example}"

    You may place more than one template function in a single file as each is called by name using the {function:file.py::function_name} format
"""

from typing import List, Union

import osxphotos
from osxphotos.phototemplate import RenderOptions

def directory(photo: osxphotos.PhotoInfo, **kwargs) -> Union[List, str]:
    """example function for {function} template; creates export directory to match Photos structure

    Args:
        photo: osxphotos.PhotoInfo object
        **kwargs: not currently used, placeholder to keep functions compatible with possible changes to {function}

    Returns:
        str or list of str of values that should be substituted for the {function} template
    """

    directories = []

    if not photo.hidden and not photo.intrash and not photo.shared:
        # set directors to [Photos/Library/year/mm/dd]
        # render_template returns a tuple of [rendered value(s)], [unmatched]
        # here, we can ignore the unmatched value, assigned to _, as we know template will match
        directories, _ = photo.render_template("Photos/Library/{created.year}/{created.mm}/{created.dd}")
    if photo.favorite:
        directories.append("Photos/Favorites")
    if photo.hidden:
        directories.append("Photos/Hidden")
    if photo.intrash:
        directories.append("Photos/Recently Deleted")

    # render the folders and albums in folder/subfolder/album format
    # the __NO_ALBUM__ is used as a sentinel to strip out photos not in an album
    folder_albums, _ = photo.render_template(
        "{folder_album}", RenderOptions(dirname=True, none_str="__NO_ALBUM__")
    )
    if folder_albums:
        root_directory = "Shared Albums/" if photo.shared else "My Albums/"
        directories.extend(
            [
                root_directory + folder_album
                for folder_album in folder_albums
                if folder_album != "__NO_ALBUM__"
            ]
        )

    return directories

Using the following command, osxphotos will maintain a synced backup of the photo library with the following directory structure.

osxphotos export /path/to/export --directory "{function:export_template.py::directory}" -V --deleted --update --cleanup --touch-file --download-missing

Photos
-- Library      (Excludes Hidden, Recently Deleted, and Shared)
---- {created.year}
------ {created.mm}
-------- {created.dd}
-- Favorites
-- Hidden
-- Recently Deleted
My Albums
-- Album 1
-- Album 2
-- Folder 1
---- Album 3
Shared Albums
-- Shared Album 1
-- Shared Album 2

A couple caveats to note:

1. Two or more albums of the same name in the same folder or base directory will be merged. Albums with the same name in different folders will not be merged.
2. The --cleanup function will not recursively delete empty folders and will only delete the lowest level empty directory on each iteration of the export command. For example, if folder Library/2021/06/21 has 1 photo that gets deleted, only the folder 21 will be deleted on the first run of the command. If you run the same command again, it will delete 06, and then run again to delete 2021 (assuming all these folders are empty after you delete the one image). This can be mitigated by running a separate python function to recursively delete any empty directories after the export command is run.

[RhetTbull]

The --cleanup function will not recursively delete empty folders and will only delete the lowest level empty directory on each iteration of the export command. For example, if folder Library/2021/06/21 has 1 photo that gets deleted, only the folder 21 will be deleted on the first run of the command. If you run the same command again, it will delete 06, and then run again to delete 2021 (assuming all these folders are empty after you delete the one image). This can be mitigated by running a separate python function to recursively delete any empty directories after the export command is run.

I'll open a new issue for this as I think the "expected" behavior for --cleanup should be to delete all directory trees that contain only empty directories.

[RhetTbull]

@mkirkland4874 I've expanded your example to also include People, Places, Imports, and Media Types and put it in the examples folder.

[RhetTbull]

@all-contributors add @mkirkland4874 for example

[allcontributors[bot]]

@RhetTbull

I've put up a pull request to add @mkirkland4874! :tada:

[RhetTbull]

@mkirkland4874 Though you've solved your immediate question, it ocurred to me there's another easy way to deal with the issue of multiple albums with the same name. If the album in question has a unique path (e.g. it's in a folder or sub-folder), you can use osxphotos' --regex feature to export only photos where a rendered template matches a regular express. For example, I have two albums named Test Album but one of them is in Folder1. The following exports only photos in the album inside Folder1:

osxphotos export ~/path/to/export -V --regex "Folder1/Test Album" "{folder_album}"

This works by evaluating the "{folder_album}" template then matching it against the pattern "Folder1/Test Album"