Closed dburner closed 1 year ago
Seems interesting!
It would help me intruduce this faster if i had some reaserch already - it this saved in exif/somewhere?
I think its saved in the json, not sure, I will do some research, and try to provide some examples.
No no, we already have info about albums from jsons
What i need to know is where to put this data inside JPGs itself
blank_page (1).zip I've uploaded a photo from Synlology with some tags inserted. Hope this helps.
EDIT: Add jpeg also. I'm trying to read this using C# but can't really find where my tags are writen.
I did some digging, found another website (https://processing.compress-or-die.com/analyze-process) that shows this image has XMP
and IPTC
tags.
Found a library to extrat these in C# https://github.com/drewnoakes/metadata-extractor-dotnet
This seems quite crazy 😄 . Never thought this would be so complicated. So apparently metadata can be in diferent formats JFIF
, XPM
and IPTC
.
private static void MetadataExtract()
{
using var stream = File.OpenRead(Filename);
var directories = JpegMetadataReader.ReadMetadata(stream);
}
@TheLastGimbus had any luck with this? 😃
i will try to add this when i will actually be inserting exifs into photos :+1:
Oh ok, I thought you were already addding exif info for timestamps.
Not yet, but soon ;)
did some looking into inserting exifs into photos
it may be very expensinve operation - to insert the exif (with currently available libraries) i will need to:
...this whole process may:
my current idea is to insert the exif only to files that don't have it already :thinking:
duplicate-copy
does ?Chiming in : I don't care about album tags, but I care about correcting the wrong EXIF data. So, if you would insert EXIF only in files which do not have some, then it's not useful for me. As far as performance is concerned, as long as I get all my files correctly in the end, I don't care if it has to run for three days in a row (though if you're moving files after processing, then the whole process is interruptible-resumable ?)
I care about correcting the wrong EXIF data
good to know :eyes: - now, I need details:
more of details: could you send/show me examples for any questions above?
as in #191 - would you mind downloading and running one exiftool command?
edit note: i saw you said:
I'm not confident enough yet to use exiftool properly
hm, okay, but it would be literally:
exiftool "-AllDates<FileModifyDate" *
- very safe and same for everyone :+1: - exiftool is wayyyy older, thus better tested - than anything i will find for Dart any time soon :1st_place_medal:far as performance is concerned, as long as I get all my files correctly in the end, I don't care if it has to run for three days in a row
that's good to know :thinking: - i'll care more about "getting files correctly" then
though if you're moving files after processing, then the whole process is interruptible-resumable ?
not at all - i first read and analize the situation read-only, then copy/move files to output, and then do anything to those files in the output (change lastModified
etc) - (i want to have consistent support for --copy
mode, where input folder is untouched - mainly for myself where this makes testing way easier)
...and, given on how many user-error and confusion i've seen, i want to keep whole gpth straightforward
...and to this "policy", if anything crashes or you want to stop and resume - you just close it and re-run all steps again
so no, no interrupt-resume
EXIF in my photos seem to be formally ok, it's their contents that is wrong. Namely, some photos received in this field the time at which the takeout was done instead of the time at which the photo was taken. I never edited this kind of things in gphotos. I care only about the time, because that's the thing that pollutes the SPhotos timeline .
Next step as per your request : I will provide an exemple and run exiftool on it, maybe more appropriate in https://github.com/TheLastGimbus/GooglePhotosTakeoutHelper/discussions/191 ?
some photos received in this field the time at which the takeout was done
What the hell??? I thouth there are no more surprises left from google ;_; ... this is terrifing
yes, please show me screenshots/exiftool dumps of such photos + their jsons etc (before running gpth/anything)
to dump all exif stuff with exiftool just run:
$ exiftool photo.jpg
ExifTool Version Number : 12.50
File Name : DSC_0270.JPG
Directory : .
File Size : 318 kB
File Modification Date/Time : 2023:03:20 09:30:40+01:00
File Access Date/Time : 0000:00:00 00:00:00
File Inode Change Date/Time : 2023:03:27 20:56:32+02:00
File Permissions : -rw-rw-r--
File Type : JPEG
..........
......
and paste it here (note that this may, obviously, share me the place where photo was took, in case you would really not want it)
maybe more appropriate in https://github.com/TheLastGimbus/GooglePhotosTakeoutHelper/discussions/191
yess, please post stuff there
What the hell??? I thouth there are no more surprises left from google ;_; ... this is terrifing
My deepest most sincere apologies, it seems I wrongly read the information from my takeouts, and that did not seem to have happened. Either no EXIF data, or EXIF createDate data that seems to match the JSON's phototaketime date.
No problem man 😅😅 good to know your photos are good
As a followup, the assumption that for every file that misses EXIF date information there would be a corresponding json file was…wrong.
Okay, running... searching for everything in input folder...
Finding duplicates...
Guessing dates from files : ........................................ 34/7549
Can't get date on Takeout/Google Photos/Photos from 2015/IMG-20150125-WA0000-modifié.jpg
Guessing dates from files : ........................................ 55/7549
Can't get date on Takeout/Google Photos/Photos from 2015/IMG-20150125-WA0001-modifié.jpg
Guessing dates from files : ........................................ 67/7549
Can't get date on Takeout/Google Photos/Photos from 2015/Constat amiable 17-01-2014 Verso-modifié.jpg
Guessing dates from files : ........................................ 73/7549
Can't get date on Takeout/Google Photos/Photos from 2015/1428073707384-modifié.jpg
Guessing dates from files : ........................................ 141/7549
Can't get date on Takeout/Google Photos/Photos from 2015/Constat amiable 17-01-2014 Recto-modifié.jpg
Can't get date on Takeout/Google Photos/Photos from 2015/IMG-20150125-WA0003-modifié.jpg
Guessing dates from files : ........................................ 142/7549
Can't get date on Takeout/Google Photos/Photos from 2015/IMG-20150125-WA0004-modifié.jpg
Guessing dates from files : ........................................ 150/7549
Can't get date on Takeout/Google Photos/Photos from 2015/IMG-20150125-WA0002-modifié.jpg
Guessing dates from files : ........................................ 170/7549
Can't get date on Takeout/Google Photos/Photos from 2012/The Wave-modifié.jpg
Guessing dates from files : ........................................ 172/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Purple Rose-modifié.jpg
Guessing dates from files : ........................................ 175/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Tribal Flames-modifié.jpg
Guessing dates from files : ........................................ 188/7549
Can't get date on Takeout/Google Photos/Photos from 2012/My Other Creation-modifié.jpg
Guessing dates from files : â–ˆ....................................... 191/7549
Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0196-modifié.jpg
Guessing dates from files : â–ˆ....................................... 192/7549
Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0191-modifié.jpg
Guessing dates from files : â–ˆ....................................... 195/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Paradise-modifié.jpg
Guessing dates from files : â–ˆ....................................... 212/7549
Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0190-modifié.jpg
Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0197-modifié.jpg
Guessing dates from files : â–ˆ....................................... 217/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Dolphin Paradise-modifié.jpg
Guessing dates from files : â–ˆ....................................... 228/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Across The Blue-modifié.jpg
Guessing dates from files : â–ˆ....................................... 230/7549
Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0188-modifié.jpg
Guessing dates from files : â–ˆ....................................... 250/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Purple Drop-modifié.jpg
Guessing dates from files : â–ˆ....................................... 260/7549
Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0189-modifié.jpg
Guessing dates from files : â–ˆ....................................... 263/7549
Can't get date on Takeout/Google Photos/Photos from 2012/My Creation-modifié.jpg
Guessing dates from files : â–ˆ....................................... 284/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Floral Butterfly-modifié.jpg
Guessing dates from files : â–ˆ....................................... 285/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Purple Night-modifié.jpg
Guessing dates from files : â–ˆ....................................... 296/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Abstract Art-modifié.jpg
Guessing dates from files : â–ˆ....................................... 299/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Abstract-modifié.jpg
Can't get date on Takeout/Google Photos/Photos from 2012/Red Dragon-modifié.jpg
Guessing dates from files : â–ˆ....................................... 303/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Surfer-modifié.jpg
Guessing dates from files : â–ˆ....................................... 310/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Fire Flower-modifié.jpg
Guessing dates from files : â–ˆ....................................... 321/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Awesome View-modifié.jpg
Guessing dates from files : â–ˆ....................................... 325/7549
Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0195-modifié.jpg
Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0192-modifié.jpg
Guessing dates from files : â–ˆ....................................... 331/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Wolf-modifié.jpg
Guessing dates from files : â–ˆ....................................... 332/7549
Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0187-modifié.jpg
Guessing dates from files : â–ˆ....................................... 340/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Nice View-modifié.jpg
Guessing dates from files : â–ˆ....................................... 347/7549
Can't get date on Takeout/Google Photos/Photos from 2012/Red Heart-modifié.jpg
Guessing dates from files : â–ˆ....................................... 349/7549
Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0186-modifié.jpg
Guessing dates from files : â–ˆ....................................... 350/7549
Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0193-modifié.jpg
Guessing dates from files : â–ˆ....................................... 351/7549
Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0194-modifié.jpg
Guessing dates from files : ███..................................... 721/7549
Can't get date on Takeout/Google Photos/Photos from 2013/.tempOrganizerImg-modifié.jpg
Guessing dates from files : ████.................................... 882/7549
Can't get date on Takeout/Google Photos/Photos from 2014/IMG_38669120553074-modifié.jpeg
Guessing dates from files : ████.................................... 917/7549
Can't get date on Takeout/Google Photos/Photos from 2014/IMG-20141225-WA0001-modifié.jpg
Guessing dates from files : █████................................... 945/7549
Can't get date on Takeout/Google Photos/Photos from 2014/IMG-20141225-WA0000-modifié.jpg
Guessing dates from files : █████................................... 1038/7549
Can't get date on Takeout/Google Photos/Photos from 2011/DSC_0076-modifié.jpg
Guessing dates from files : █████................................... 1060/7549
Can't get date on Takeout/Google Photos/Photos from 2011/DSC_0074-modifié.jpg
Guessing dates from files : █████................................... 1067/7549
Can't get date on Takeout/Google Photos/Photos from 2011/DSC_0079-modifié.jpg
Guessing dates from files : █████................................... 1073/7549
Can't get date on Takeout/Google Photos/Photos from 2011/DSC_0078-modifié.jpg
Guessing dates from files : █████████████........................... 2458/7549
Can't get date on Takeout/Google Photos/Photos from 2020/IMG-be001af48da68a6588105cde09d690da-V-modifié.jpg
Guessing dates from files : █████████████........................... 2561/7549
Can't get date on Takeout/Google Photos/Photos from 2020/IMG-9a04b76a6027fb5bd163d5cc2e33c8c8-V-modifié.jpg
Guessing dates from files : ██████████████.......................... 2769/7549
Can't get date on Takeout/Google Photos/Photos from 2020/IMG-2a6602a48d4da9bbaed7f0c2f7687b51-V-modifié.jpg
Guessing dates from files : ████████████████........................ 3095/7549
Can't get date on Takeout/Google Photos/Photos from 2020/IMG-f1f40898bdf5fd31cd1eaa92ae2d6196-V-modifié.jpg
Guessing dates from files : ██████████████████████.................. 4288/7549
Can't get date on Takeout/Google Photos/Photos from 2018/IMG-61566c4e2c032528b0eb4d50aa901d15-V-modifié.jpg
Guessing dates from files : ██████████████████████████████████████.. 7317/7549
Can't get date on Takeout/Google Photos/Photos from 2017/BlurImage_13-4-2017-2-24-21-modifié.jpg
There are corresponding un-modifiéd files in the same original folders, so I'll try to manually touch
the ALL_PHOTOS/date-unknown
files. Unless you've got a better, faster suggestion.
those are "extras" - modified/effects photos - they often have jsons without the -modified
suffix - look https://github.com/TheLastGimbus/GooglePhotosTakeoutHelper/blob/master/lib/extras.dart
we can just add your french "-modifié" and handle them as well :rocket:
As a followup, the assumption that for every file that misses EXIF date information there would be a corresponding json file was…wrong.
Okay, running... searching for everything in input folder... Finding duplicates... Guessing dates from files : ........................................ 34/7549 Can't get date on Takeout/Google Photos/Photos from 2015/IMG-20150125-WA0000-modifié.jpg Guessing dates from files : ........................................ 55/7549 Can't get date on Takeout/Google Photos/Photos from 2015/IMG-20150125-WA0001-modifié.jpg Guessing dates from files : ........................................ 67/7549 Can't get date on Takeout/Google Photos/Photos from 2015/Constat amiable 17-01-2014 Verso-modifié.jpg Guessing dates from files : ........................................ 73/7549 Can't get date on Takeout/Google Photos/Photos from 2015/1428073707384-modifié.jpg Guessing dates from files : ........................................ 141/7549 Can't get date on Takeout/Google Photos/Photos from 2015/Constat amiable 17-01-2014 Recto-modifié.jpg Can't get date on Takeout/Google Photos/Photos from 2015/IMG-20150125-WA0003-modifié.jpg Guessing dates from files : ........................................ 142/7549 Can't get date on Takeout/Google Photos/Photos from 2015/IMG-20150125-WA0004-modifié.jpg Guessing dates from files : ........................................ 150/7549 Can't get date on Takeout/Google Photos/Photos from 2015/IMG-20150125-WA0002-modifié.jpg Guessing dates from files : ........................................ 170/7549 Can't get date on Takeout/Google Photos/Photos from 2012/The Wave-modifié.jpg Guessing dates from files : ........................................ 172/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Purple Rose-modifié.jpg Guessing dates from files : ........................................ 175/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Tribal Flames-modifié.jpg Guessing dates from files : ........................................ 188/7549 Can't get date on Takeout/Google Photos/Photos from 2012/My Other Creation-modifié.jpg Guessing dates from files : █....................................... 191/7549 Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0196-modifié.jpg Guessing dates from files : █....................................... 192/7549 Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0191-modifié.jpg Guessing dates from files : █....................................... 195/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Paradise-modifié.jpg Guessing dates from files : █....................................... 212/7549 Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0190-modifié.jpg Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0197-modifié.jpg Guessing dates from files : █....................................... 217/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Dolphin Paradise-modifié.jpg Guessing dates from files : █....................................... 228/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Across The Blue-modifié.jpg Guessing dates from files : █....................................... 230/7549 Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0188-modifié.jpg Guessing dates from files : █....................................... 250/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Purple Drop-modifié.jpg Guessing dates from files : █....................................... 260/7549 Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0189-modifié.jpg Guessing dates from files : █....................................... 263/7549 Can't get date on Takeout/Google Photos/Photos from 2012/My Creation-modifié.jpg Guessing dates from files : █....................................... 284/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Floral Butterfly-modifié.jpg Guessing dates from files : █....................................... 285/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Purple Night-modifié.jpg Guessing dates from files : █....................................... 296/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Abstract Art-modifié.jpg Guessing dates from files : █....................................... 299/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Abstract-modifié.jpg Can't get date on Takeout/Google Photos/Photos from 2012/Red Dragon-modifié.jpg Guessing dates from files : █....................................... 303/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Surfer-modifié.jpg Guessing dates from files : █....................................... 310/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Fire Flower-modifié.jpg Guessing dates from files : █....................................... 321/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Awesome View-modifié.jpg Guessing dates from files : █....................................... 325/7549 Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0195-modifié.jpg Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0192-modifié.jpg Guessing dates from files : █....................................... 331/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Wolf-modifié.jpg Guessing dates from files : █....................................... 332/7549 Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0187-modifié.jpg Guessing dates from files : █....................................... 340/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Nice View-modifié.jpg Guessing dates from files : █....................................... 347/7549 Can't get date on Takeout/Google Photos/Photos from 2012/Red Heart-modifié.jpg Guessing dates from files : █....................................... 349/7549 Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0186-modifié.jpg Guessing dates from files : █....................................... 350/7549 Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0193-modifié.jpg Guessing dates from files : █....................................... 351/7549 Can't get date on Takeout/Google Photos/Photos from 2012/DSC_0194-modifié.jpg Guessing dates from files : ███..................................... 721/7549 Can't get date on Takeout/Google Photos/Photos from 2013/.tempOrganizerImg-modifié.jpg Guessing dates from files : ████.................................... 882/7549 Can't get date on Takeout/Google Photos/Photos from 2014/IMG_38669120553074-modifié.jpeg Guessing dates from files : ████.................................... 917/7549 Can't get date on Takeout/Google Photos/Photos from 2014/IMG-20141225-WA0001-modifié.jpg Guessing dates from files : █████................................... 945/7549 Can't get date on Takeout/Google Photos/Photos from 2014/IMG-20141225-WA0000-modifié.jpg Guessing dates from files : █████................................... 1038/7549 Can't get date on Takeout/Google Photos/Photos from 2011/DSC_0076-modifié.jpg Guessing dates from files : █████................................... 1060/7549 Can't get date on Takeout/Google Photos/Photos from 2011/DSC_0074-modifié.jpg Guessing dates from files : █████................................... 1067/7549 Can't get date on Takeout/Google Photos/Photos from 2011/DSC_0079-modifié.jpg Guessing dates from files : █████................................... 1073/7549 Can't get date on Takeout/Google Photos/Photos from 2011/DSC_0078-modifié.jpg Guessing dates from files : █████████████........................... 2458/7549 Can't get date on Takeout/Google Photos/Photos from 2020/IMG-be001af48da68a6588105cde09d690da-V-modifié.jpg Guessing dates from files : █████████████........................... 2561/7549 Can't get date on Takeout/Google Photos/Photos from 2020/IMG-9a04b76a6027fb5bd163d5cc2e33c8c8-V-modifié.jpg Guessing dates from files : ██████████████.......................... 2769/7549 Can't get date on Takeout/Google Photos/Photos from 2020/IMG-2a6602a48d4da9bbaed7f0c2f7687b51-V-modifié.jpg Guessing dates from files : ████████████████........................ 3095/7549 Can't get date on Takeout/Google Photos/Photos from 2020/IMG-f1f40898bdf5fd31cd1eaa92ae2d6196-V-modifié.jpg Guessing dates from files : ██████████████████████.................. 4288/7549 Can't get date on Takeout/Google Photos/Photos from 2018/IMG-61566c4e2c032528b0eb4d50aa901d15-V-modifié.jpg Guessing dates from files : ██████████████████████████████████████.. 7317/7549 Can't get date on Takeout/Google Photos/Photos from 2017/BlurImage_13-4-2017-2-24-21-modifié.jpg
There are corresponding un-modifiéd files in the same original folders, so I'll try to manually
touch
theALL_PHOTOS/date-unknown
files. Unless you've got a better, faster suggestion.
For the record, it involves the following steps for each file F-modifié.jpg
in date-unknown : (CWD : gpth-output folder)
find . -name F\* -ls
touch -r ALL_PHOTOS/yearfound/monthfound/F.jpg ALL_PHOTOS/date-unknown/F-modifié.jpg
mv ALL_PHOTOS/date-unknown/F-modifié.jpg ALL_PHOTOS/yearfound/monthfound/
we can just add your french "-modifié" and handle them as well 🚀
That would be great for my next Takeout ! :)
here, grab the nightly with those added :)
https://github.com/TheLastGimbus/GooglePhotosTakeoutHelper/actions/runs/4585888986
:o turned out i had a function for matching something-edited.jpg
in any language, but it didn't support accent characters because of regex :laughing: - added this in a0dd2842
now check it out :) - https://github.com/TheLastGimbus/GooglePhotosTakeoutHelper/actions/runs/4588597120
@dburner looking at this, for now, i don't thing im gonna touch the exifs myself, sorry :( - but, with the json
album solution, i think you (or even me) could easily write a script that would do that - for example in python, where there actually are some good exif libraries :eyes:
Oke thanks for the help.
But I do agree with @palijn, if the process takes some time I dont really care.
Maybe you can add this feature as an opt-in, maybe add an argument --add-tags
. I think this feature would fit the GooglePhotosTakeoutHelper
project.
I agree and would love to add it, but there's nothing good for exif for Dart
Maybe i can make some scripts/
folder with stuff like that...
Ok, maybe I could help out with a C# plugin that you could call as an opt in (either separate app or dynamic library).
You mentioned earier that the album information is present in the json files. I can not find any json that specifies if an image is in a particular album or not. Can you point it out for me ?
Thanks.
the json file that gpth makes with --albums json
option
Ooooo I thought google made a json that specified albums.
Does GooglePhotosTakeoutHelper
need a full google photos export to output this?
Thanks to latest updates, no :partying_face: - gpth can process whatever you give it - if you export just year folders, it will give you "one big folder", and if you give just the albums, it should handle them just as nice :+1:
Looking at this again...
a photo that i've added tags to from Synology Photos:
huh - at first i thought that this is synology's way for albums, but now i understand that this are just some tags that you added...
as such, i don't see much benefit in doing this :shrug:
at least for now where exifs are still no-no - so closing this for now...
Description
Add album metadata in image exif as a tag. I really don't want to lose album information when moving from Google Photos to Synology Photos
Example
From
https://jimpl.com/