Timeline shows images taken on one day as taken on two sequential days

[blackshirtcoder]

Canon EOS R7

I took 19 photos on February 22nd over a period of about 15 minutes. The camera assigned sequence numbers 48 through 66 to these photos.

When I open Rapid Photo Downloader, the timeline says 14 of the images were taken on the 22nd, and 5 of the images were taken on the 23rd (one day in the future). The images apparently from the future have sequence numbers 50, 53, 59, 60 and 65. Not sequential!

The time stamps reported by the file system show all files created on the 22nd (well, that's actually the "date modified" time). I've also used 'exiftool' to look at the EXIF data for each file. All 19 files have "File Access Date/Time" and "File Inode Change Date/Time" fields showing Feb 23rd, but every other time stamp in the EXIF output, including the GPS time stamps where available, show Feb 22nd. I don't see anything different in any of the time stamp fields for the photos supposedly taken on the 23rd compared to those supposedly taken on the 22nd.

I don't understand why Rapid Photo Downloader thinks 5 of these files, apparently randomly, were taken the next day. There are over 300 EXIF fields reported per image. It's quite likely I'm missing something. Anyone have any ideas?

[damonlynch]

Rapid Photo Downloader relies on Exiv2 and ExifTool to read photo and video metadata. What this means is that you have almost certainly found a bug in one of those two programs. To know what happened in your particular case you need to include the log files. See this for instructions.

[blackshirtcoder]

Yep. Sorry. Forgot that.

rpd-bug-report-20240223.tar.gz

[damonlynch]

Thanks for that. I need you to install exiv2 and then give me the output of exiv2 --version

[blackshirtcoder]

The short answer is "version 0.27.5". exiv2 was not previously installed on my system. Here is the entire transaction...

$ sudo apt install exiv2 Reading package lists... Done Building dependency tree... Done Reading state information... Done The following NEW packages will be installed: exiv2 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. Need to get 104 kB of archives. After this operation, 286 kB of additional disk space will be used. Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 exiv2 amd64 0.27.5-3ubuntu1 [104 kB] Fetched 104 kB in 0s (248 kB/s) Selecting previously unselected package exiv2. (Reading database ... 368184 files and directories currently installed.) Preparing to unpack .../exiv2_0.27.5-3ubuntu1_amd64.deb ... Unpacking exiv2 (0.27.5-3ubuntu1) ... Setting up exiv2 (0.27.5-3ubuntu1) ... Processing triggers for man-db (2.10.2-1) ...

$ exiv2 --version exiv2 0.27.5

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA

[damonlynch]

Thanks. Now, do this with one of your CR3 files in which the date time is wrong:

exiv2 path/to/your/CR3 exiftool path/to/your/CR3

and compare the output, to see which is correct. One of date time outputs is likely to be incorrect. Report back here with the results.

[blackshirtcoder]

I am afraid we have probably reached the end of the line on this, and I've wasted your time.

I did as you instructed, and neither tool produced output that had incorrect dates. As mentioned previously, on the day this happened, I was examining the output of exiftool for both "good" and "bad" CR3 files in an effort to find a mismatch somewhere. I didn't see any obvious differences between good and bad files, and the only dates that had a day field as "23" (the day I was copying files over) as opposed to "22" (the day the images were taken) were these two fields...

File Access Date/Time File Inode Change Date/Time

I assume these must come from the file system, and not the image metadata. At any rate, all other timestamps in the exiftool output that I could recognize as timestamps (as opposed to, say, a time_t value) had the day as the 22nd, which is correct. This morning I ran exiv2 against a candidate "good" and "bad" file, and both reported an "Image timestamp" field from the 22nd -- correct.

It's worth noting that the output of exiv2 is 25 lines long, whereas the output from exiftool is 330 lines long, so it is a lot easier to miss something in the exiftool output.

But that's neither here nor there. Here is the problem. I have all the original CR3 files on my NAS box as they were after RPD copied them over from my camera, but I DON'T have the originals on the camera's memory card any longer. As a test, this morning I copied all the CR3s from the session in question over to my local drive, and then used RPD to process them again, this time sending them to a different location on my NAS box. Guess what? RPD reports all 19 files as having come from the 22nd, which is correct.

I also ran both "good" and "bad" CR3s from where I'd placed them on my local machine back through exiv2 and exiftool and, like before, both tools report all timestamps with a day value of "22" with the exception of the two exiftool fields mentioned above, which today are reporting "27" (today).

So the problem seems to have occurred when copying the CR3s directly from my camera (via USB), but NOT when copying those ostensibly identical files from my local Linux drive. I have no way to go back and try to re-copy the files directly from my camera, because I reformatted the original memory card after copying the images to my NAS box. So the problem seems to be related to copying the images directly from my camera, and if that problem can't be reproduced, I don't know how you could ever get to the bottom of it.

I did notice something unusual with the file timestamps reported by Caja and "ls -l" from the command line as it relates to the timestamps on the camera's memory card. Both utilities reported the times to be at 2 in the morning, when the pictures were actually taken at 10 in the morning. This 8 hour difference corresponds to my location 8 hours west of the prime meridian. Digging into it, I discovered that Linux interprets file timestamps on FAT32 formatted media as UTC times, even though the camera wrote them as "local times". Now, neither the local time or the UTC version of that timestamp occurred anywhere around midnight, so my original problem can not be attributed to UTC rolling over from one day to the next. Besides, all photos were taken during the same hour, and the "good" images and the "bad" images are interleaved, rather than sequential as they would be if this was a clock rollover problem. So I'm not suggesting my original problem is a clock rollover problem. I'm just pointing out there are some oddities when it comes to Linux interpreting FAT32 timestamps.

Also, some of the 19 images I captured on the morning of the 22nd had embedded GPS data, and some did not, however, there is no correlation between "good" files and "bad" files and whether or not they had GPS data. I have "good" and "bad" files from both cases. For those files that do contain GPS data, the GPS timestamp is correct -- Around 16:00 on the 22nd, UTC.

I do appreciate your time and attention to this matter. I know you would like to get to the bottom of it if you could, but I feel like we've probably reached a dead end.

[damonlynch]

Thanks for looking into this more, and your explanation. Do you have thumbnail generation turned on?

[blackshirtcoder]

Under the "Thumbnails" section of the "Preferences" dialog box, I do have "Generate thumbnails" checked, but "Cache thumbnails" and "Generate system thumbnails" are both unchecked.

[damonlynch]

The reason I ask is because generating the Timeline is non-trivial. To build an accurate timeline, the photo and video metadata needs to be read. However, reading this metadata is very slow. So initially the Timeline is built using the file timestamps, and updated when thumbnailing occurs (or the download occurs).

It's possible something went wrong where the Timeline was not refreshed after the thumbnailing occurred. However that should happen rarely. Probably the way the program tracks its internal state with respect to refreshing the Timeline (among other things) is not 100% correct under all circumstances, which is not surprising, because it's actually quite complicated.

And yes the fact that FAT32 does not include timezones in its timestamps is a major pain in the butt. Some design choices that Microsoft made back in the 80s continue to haunt us today. Maybe they seemed wise at the time, but they sure are painful today.