Sort by date does not does not correctly handle timezones

[mcdamo]

Sort by date does not does not correctly handle timezones.

In the case that a directory or search contains media with and without timezone data they will not display in the expected order.

Cameras generally store the Local Time that the media was taken but modern cameras or smartphones can now record the Timezone offset as well.

The problem arising with sorting in Pigallery appears because all media is converted to UTC during indexing.

For example two photos taken at the same time, midday in GMT+09, one on a camera and the other on a smartphone

1. on a camera without timezone data will be indexed as 12:00 with 0 offset
2. on a smartphone with timezone will be indexed as 03:00 with +09:00 offset.

This means that the first photo will be sorted as if it were 9 hours later than the second within Pigallery.

Here in the code the sorting services uses the base UTC time without referencing the offset: https://github.com/bpatrik/pigallery2/blob/6c53aca7e7d5b65d257c80737647d57854592bbe/src/frontend/app/ui/gallery/navigator/sorting.service.ts#L153

It may be more coherent to sort all media by the Local Time instead of UTC.

[bpatrik]

Thank you for the report.

@grasdk do you have a quick insight to this issue?

[grasdk]

@bpatrik: I think @mcdamo already found the problem (or part of it).

It appears that I missed sorting.service.ts, when adding comments about searching and filtering should take offset into account, when I introduced offset to the metadata.

I wrote some //TODOs in the searchmanager and in the filter.service.ts, but not in sorting.service.ts.

Basically, one can do something like

return (a.metadata.creationDate + Utils.getOffsetMinutes(a.metadata.creationDateOffset)*60000) - (b.metadata.creationDate + Utils.getOffsetMinutes(b.metadata.creationDateOffset)*60000)

in the line that @mcdamo writes.

This will take the offset as ms (minutes*60000) and add it to the creationDates before comparing. Addition should work fine with negative offsets. There are some edge cases, where either or both offsets are not defined, that needs to be handled, in or before the line I suggested.

[grasdk]

Hey @mcdamo and @bpatrik

I had a look at this today. I guess the real solution here is a configuration, perhaps an omnipresent ui element that will allow you to switch between: picture's local time and fixed timezone (and fixed timezone can be configured in the config to UTC or any timezone you prefer).

Right now it's a mix. The deep down representation of a photo's timestamp is UTC. This is used for sorting in the line you discovered above.

Some months ago I added the offset information to this, so you can calculate the picture's local time. This is shown in the info of pictures and it seems to be in effect in the list view of images as well. So I guess it's a bit inconsistent and I can see that in some cases, you would like sorting to function on picture's local time instead.

• @mcdamo: For your particular case, as you also write. Your incomplete metadata causes the problem with sorting. One camera stores no timezone offset and Pigallery2 can't guess that. I would probably fix the metadata with exiftool or the like. It would also be solved if you had the switch I suggested above. -- I will try and work on the code to make this possible in the coming time, though I'm not good with all the ui elements. But I could make the functionality ready and perhaps someone else would make the UI stuff (or I could learn) :)

---

PS: I'm attaching some pictures I generated for showing the problem. If we pretend they are photos taken all over the world, they are taken within 4 minutes in total, but have local time with great differences. Right now they're sorted in UTC/Global time. It makes sense in this scenario: If 7 friends talk over the Internet, asking each other to take photos of their location and all upload them to the same storage via a service that doesn't ruin the metadata, they would show up in the exact order they were taken! Sorting in local time would change that a lot. :)

Here is a screenshot of how pigallery sorts them now:

Here are the pictures - feel free to use:

[mcdamo]

@grasdk I think the example of multiple people from different timezones submitting photos to a single album is probably not a common use case for pigallery?

Deep down in the EXIF data a photo's timestamp is a date string representing when the photo was taken. I believe this is commonly local-time that the camera is configured in, though the spec is ambiguous.

The timezone offset was not added to the EXIF spec until v2.31. While this spec was reportedly released in mid 2016, it was many years after that manufacturers started using.

For a real world example a smartphone I tested added the offset tag via software update in 2019, while a 2021-camera I tested did not initially support the offset tag and later added it via firmware update in 2023. Earlier make cameras simply do not use this offset tag.

The issue with pigallery converting all image metadata to UTC is that anyone using any images or videos without offset tags this is going to cause ongoing confusion and problems.

I would prefer to completely ignore the timezone offset every where that metadata is used, including searching, filtering, and sorting. In effect always using the photo's recorded time and never converting between timezones.

[grasdk]

A fair point and opinion. I think we must agree to disagree on what 'common' is.

I'm looking at offset as an important piece of data that should be taken into account. Of course if your pictures are not homogeneous (all having offset or all missing offset), this will lead to problems.

Personally, when I get pictures that lack any tag that I'm interested in (tags, timestamps, offsets, toponomy names, gps), I manually add them to the picture metadata rather than expecting tools to "guess it". Geosetter, DigiKam and exiftool are among the tools I use for this.

However, everyone is not as rigid in metadata homogeneity, so I will work on making a configuration for PiGallery so you can configure it to ignore offset. This will be my next focus for contribution to PiGallery. :-)

[bpatrik]

@mcdamo

@grasdk I think the example of multiple people from different timezones submitting photos to a single album is probably not a common use case for pigallery?

One would think so, but I'm pretty sure that if we make this assumption, issues will appear like "I'm using pigallery with a single folder. I love it, but now its broken". I have seen this before.

On the broader topic: I agree that there should be a clear way how pigalley handles time. (this is not a simple problem actually. timezones are hard). I think having now supporting offsets is a way forward. I think a general user experience should be that user can see their photos in the same circumstance as it was taken. Eg: if it was taken in NY noon, using the app in London, the app should still show noon. Being able to see the photo with relative time to our current timezone also has value. Especially if timezones are mixing in one folder. Search, filtering, sorting should also "as user expects"

I think use should be able to switch between different timezone settings. (as @grasdk mentions in prev. comment)

[grasdk]

@mcdamo I would say this bug is resolved in the latest edge version, if you leave the timestamp offset setting on "ignore". See https://github.com/bpatrik/pigallery2/pull/902

[mcdamo]

Thanks @grasdk, this option resolves the sorting of images without timezones.