A tool to populate missing DateTimeOriginal
EXIF metadata in Google Photos takeout, using Google's JSON metadata.
Example usage:
yarn
yarn start --inputDir ~/takeout --outputDir ~/output --errorDir ~/error
I wrote this tool to help me overcome some issues that I had when trying to make use of photos exported from Google Photos using Google Takeout.
My goal was to extract all photos from my Google Photos account and incorporate them into a master photo library on my Mac. This library would be organised into a date-based folder structure, with images being automatically moved into the correct structure using Silent Sifter.
Silent Sifter provides a fast way to organise images into folders based on the timestamps embedded in the image metadata or failing that, the file modification timestamps.
Whilst it is great that I was able to use Google Takeout to extract all of my stored images from Google Photos at once, I found that some images were landing in the wrong place due to missing DateTimeOriginal
EXIF timestamps.
This tool aims to eliminate some of those issues by reading the photoTakenTime
timestamp from the JSON metadata files that are included in Google Takeout export and using it to:
DateTimeOriginal
field in the EXIF metadata if this field is not already set At the time of writing (October 2020), Google Takeout provides you with one or more zip files, structured in a way that is fairly unintuitive and tricky to make use of directly.
Extracting the zip, you might find something similar to this:
Extracted Takeout Zip
Google Photos
2006-01-01
IMG0305.jpg.json
2020-09-01
IMG1001.jpg
IMG1001.jpg.json
2020-09-02
IMG1002.jpg.json
metadata.json
2020-09-04
IMG1003.jpg
IMG1003.json
SomeAlbumName
IMG1004.jpg
IMG1004.jpg.json
IMG1005.jpg
IMG1005.json
archive_browser.html
There are some interesting challenges to note here:
photoTakenTime
property.IMG123.jpg
, sometimes you get IMG123.jpg.json
but sometimes it's just IMG123.json
The first step to using this tool is to request & download a Google Takeout
. At the time of writing the steps to do this are:
Google Photos
All photo albums included
.inputDir
. The tool takes in three parameters:
inputDir
directory path containing the extracted Google Takeout.outputDir
directory path where processed files will be moved to. This needs to be an empty directory and can be anywhere on the disk. errorDir
directory path where images with bad EXIF data that fail to process will be moved to. The folder can be empty.The inputDir
needs to be a single directory containing an extracted zip from Google takeout. As described in the section above, it is important that the zip has been extracted into a directory (this tool doesn't extract zips for you) and that it is a single folder containing the whole Takeout (or if coming from multiple archives, that they have been properly merged together).
For example:
Takeout
Google Photos
2017-04-03
2020-01-01
...
In order to avoid touching files that are not photos or videos, this tool will only process files whose extensions are whitelisted in the configuration options. Any other files will be ignored and not included in the output.
To customise which files are processed, edit the src/config.ts
file to suit your needs. For each extension you can also configure whether or not to attempt to read/write EXIF metadata for that file type.
The default configuration is as follows:
┌──────────┬─────────┐
│Extension │EXIF │
├──────────┼─────────┤
│.jpeg │true │
│.jpg │true │
│.heic │true │
│.gif │false │
│.mp4 │false │
│.png │false │
│.avi │false │
│.mov │false │
└──────────┴─────────┘
The tool will do the following:
Find all "media files" with one of the supported extensions (see "Configuring supported file types" above) from the (nested) inputDir
folder structure.
For each "media file":
a. Look for a corresponding sidecar JSON metadata file (see the section below for more on this) and if found, read the photoTakenTime
field
b. Copy the media file to the output directory
c. Update the file modification date to the photoTakenTime
found in the JSON metadata
d. If the file supports EXIF (e.g. JPEG images), read the EXIF metadata and write the DateTimeOriginal
field if it does not already have a value in this field
e. If an error occurs whilst processing the file, copy it to the directory specified in the errorDir
argument, so that it can be inspected manually or removed
Display a summary of work completed
The Google Takeout file/folder structure has some interesting inconsistencies/quirks which make it tricky to work with.
This tool makes a best-effort attempt to work with this structure. Read on to get a feel for how it works.
Imagine you have an image named foo.jpg
.
In the same directory, we also expect to find a JSON file containing metadata about the image. The pattern for how this is names seems inconsistent but in general, we seem to get either: foo.json
or foo.jpg.json
. This tool will look for either of those two files.
It appears that images that were edited using Google Photos (e.g. rotated or edited on a mobile device) don't get their own JSON sidecar files.
For example, for image file foo-edited.jpg
we don't get foo-edited.json
nor foo-edited.jpg.json
. Instead we must rely on the JSON file from the original image.
To work around this, for any images with a suffix of -edited
, this tool will ignore that suffix and look for the JSON file from the original image, e.g. foo.json
or foo.jpg.json
.
I found that some of my images were named with a numeric suffix, such as foo(1).jpg
.
Counter-intuitively, the corresponding JSON file for this doesn't follow the same pattern. Instead of foo(1).json
or foo(1).jpg.json
, it is instead named foo.jpg(1).json
.
To support that, this tool will also check for files that have a number suffix in brackets and where that is the case, look for a JSON file with the pattern above.
This tool was only written for the purpose of solving my own personal requirements.
I decided to make this public on GitHub because:
With that said, please bear in mind that this tool won't be actively maintained and your mileage may vary. I'm sure it's far from perfect so if you choose to use it please proceed with caution and be careful to verify the results! I hope it's helpful.