Use of UTC for video files

mcwieger commented 1 year ago

Video files do not appear to use UTC, but seem to display the GMT timezone value.

I set the following (Quicktime) tags to make sure all possible dates used for videos are correct:

[System]        FileModifyDate                  : 2023:04:21 18:53:30+02:00
[System]        FileAccessDate                  : 2023:07:27 19:58:17+02:00
[System]        FileCreateDate                  : 2023:04:21 18:53:30+02:00
[QuickTime]     CreateDate                      : 2023:04:21 18:53:30+02:00
[QuickTime]     ModifyDate                      : 2023:04:21 18:53:30+02:00
[ItemList]      ContentCreateDate               : 2023:04:21 18:53:30+02:00
[Track1]        TrackCreateDate                 : 2023:04:21 18:53:30+02:00
[Track1]        TrackModifyDate                 : 2023:04:21 18:53:30+02:00
[Track1]        MediaCreateDate                 : 2023:04:21 18:53:30+02:00
[Track1]        MediaModifyDate                 : 2023:04:21 18:53:30+02:00
[Track2]        TrackCreateDate                 : 2023:04:21 18:53:30+02:00
[Track2]        TrackModifyDate                 : 2023:04:21 18:53:30+02:00
[Track2]        MediaCreateDate                 : 2023:04:21 18:53:30+02:00
[Track2]        MediaModifyDate                 : 2023:04:21 18:53:30+02:00

All times are set to 18:53:30 with my timezone. However, date in PiGallery2 shows 16:53:30. Can UTC be implemented for video files as it was done for pictures in https://github.com/bpatrik/pigallery2/pull/480 and https://github.com/bpatrik/pigallery2/issues/469?

Environment

OS: Synology DSM6.2
Browser: Chrome

Used app version

Docker Nightly Alpine v1.9.6-nightly

bpatrik commented 1 year ago

I think the problem lies with the metadata parsing part: The creationDate is not set correctly most likely here: https://github.com/bpatrik/pigallery2/blob/3e6bb7bff719f2c768c8ddf939cc63089a8f7776/src/backend/model/threading/MetadataLoader.ts#L20-L93

My guess would be that metadata.creationDate = stat.mtime.getTime(); is not using the right timezone maybe.

mcwieger commented 1 year ago

This is what Phil Harvey says: https://exiftool.org/forum/index.php?topic=15052.0

The Quicktime creation date indeed is not using the right timezone. If that field can be made to use UTC, the time will be correct.

bpatrik commented 1 year ago

Ahh so its Quicktime specific. Can you send me a file with what I can reproduce? Like do you have the same issue on these files (random Google search): https://file-examples.com/index.php/sample-video-files/sample-mov-files-download/

mcwieger commented 1 year ago

https://github.com/bpatrik/pigallery2/assets/8181621/0a572742-2d17-4d44-b46e-27e50dc9e285

https://github.com/bpatrik/pigallery2/assets/8181621/0082b502-e11c-46ed-8e24-dd580a128209

https://github.com/bpatrik/pigallery2/assets/8181621/cdc8adb6-f9a8-4b1d-a024-3b53f884e04a

https://github.com/bpatrik/pigallery2/assets/8181621/436c9a43-4a4c-4039-9cd9-98f5dbc07473

I've attached four files:

MOV processed with EXIFTool as UTC
MOV processed with EXIFTool as non-UTC
MP4 processed with EXIFTool as UTC
MP4 processed with EXIFTool as non-UTC

UTC was processed as follows (to show which fields I updated): exiftool.exe -P -v2 "-quicktime:CreateDate<FileModifyDate" "-quicktime:ModifyDate<FileModifyDate" "-quicktime:TrackCreateDate<FileModifyDate" "-quicktime:TrackModifyDate<FileModifyDate" "-quicktime:MediaCreateDate<FileModifyDate" "-quicktime:MediaModifyDate<FileModifyDate" "-quicktime:ContentCreateDate<FileModifyDate" <filename>

Non-UTC was processed as follows (to show which fields I updated): exiftool.exe -P -v2 "-quicktime:CreateDate<FileModifyDate" "-quicktime:ModifyDate<FileModifyDate" "-quicktime:TrackCreateDate<FileModifyDate" "-quicktime:TrackModifyDate<FileModifyDate" "-quicktime:MediaCreateDate<FileModifyDate" "-quicktime:MediaModifyDate<FileModifyDate" "-quicktime:ContentCreateDate<FileModifyDate" -api QuickTimeUTC=1 <filename>

Results in my instance:

MOV processed with EXIFTool as UTC - incorrect (one hour early)
MOV processed with EXIFTool as non-UTC - correct
MP4 processed with EXIFTool as UTC - incorrect (one hour early)
MP4 processed with EXIFTool as non-UTC - correct

bpatrik commented 1 year ago

a - mov_sample_non-UTC.2023-01-01_12-00-00 b - mov_sample_UTC.2023-01-01_12-00-00 c - mp4_sample_non-UTC.2023-01-01_12-00-00.mp4 d - mp4_sample_UTC.2023-01-01_12-00-00.mp4

What Windows sees: kép

what pigallery2 sees:

------------a.mov------------------
raw data:
{
  index: 0,
  codec_name: 'h264',
  codec_long_name: 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10',
  profile: 'High',
  codec_type: 'video',
  codec_time_base: '1/60',
  codec_tag_string: 'avc1',
  codec_tag: '0x31637661',
  width: 1920,
  height: 1080,
  coded_width: 1920,
  coded_height: 1088,
  has_b_frames: 2,
  sample_aspect_ratio: '1:1',
  display_aspect_ratio: '16:9',
  pix_fmt: 'yuv420p',
  level: 40,
  color_range: 'unknown',
  color_space: 'unknown',
  color_transfer: 'unknown',
  color_primaries: 'unknown',
  chroma_location: 'left',
  field_order: 'unknown',
  timecode: 'N/A',
  refs: 1,
  is_avc: 'true',
  nal_length_size: 4,
  id: 'N/A',
  r_frame_rate: '30/1',
  avg_frame_rate: '30/1',
  time_base: '1/15360',
  start_pts: 0,
  start_time: 0,
  duration_ts: 461322,
  duration: 30.033984,
  bit_rate: 447244,
  max_bit_rate: 'N/A',
  bits_per_raw_sample: 8,
  nb_frames: 901,
  nb_read_frames: 'N/A',
  nb_read_packets: 'N/A',
  tags: {
    creation_time: '2023-01-01T12:00:00.000000Z',
    language: 'eng',
    handler_name: 'DataHandler',
    encoder: 'Lavc57.16.101 libx264'
  },
  disposition: {
    default: 1,
    dub: 0,
    original: 0,
    comment: 0,
    lyrics: 0,
    karaoke: 0,
    forced: 0,
    hearing_impaired: 0,
    visual_impaired: 0,
    clean_effects: 0,
    attached_pic: 0,
    timed_thumbnails: 0
  }
}
extracted:
{
  size: { width: 1920, height: 1080 },
  bitRate: 447244,
  duration: 30033,
  creationDate: 1672574400000,
  fileSize: 2247292,
  fps: 30
}
------------b.mov------------------
raw data:
{
  index: 0,
  codec_name: 'h264',
  codec_long_name: 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10',
  profile: 'High',
  codec_type: 'video',
  codec_time_base: '1/60',
  codec_tag_string: 'avc1',
  codec_tag: '0x31637661',
  width: 1920,
  height: 1080,
  coded_width: 1920,
  coded_height: 1088,
  has_b_frames: 2,
  sample_aspect_ratio: '1:1',
  display_aspect_ratio: '16:9',
  pix_fmt: 'yuv420p',
  level: 40,
  color_range: 'unknown',
  color_space: 'unknown',
  color_transfer: 'unknown',
  color_primaries: 'unknown',
  chroma_location: 'left',
  field_order: 'unknown',
  timecode: 'N/A',
  refs: 1,
  is_avc: 'true',
  nal_length_size: 4,
  id: 'N/A',
  r_frame_rate: '30/1',
  avg_frame_rate: '30/1',
  time_base: '1/15360',
  start_pts: 0,
  start_time: 0,
  duration_ts: 461322,
  duration: 30.033984,
  bit_rate: 447244,
  max_bit_rate: 'N/A',
  bits_per_raw_sample: 8,
  nb_frames: 901,
  nb_read_frames: 'N/A',
  nb_read_packets: 'N/A',
  tags: {
    creation_time: '2023-01-01T11:00:00.000000Z',
    language: 'eng',
    handler_name: 'DataHandler',
    encoder: 'Lavc57.16.101 libx264'
  },
  disposition: {
    default: 1,
    dub: 0,
    original: 0,
    comment: 0,
    lyrics: 0,
    karaoke: 0,
    forced: 0,
    hearing_impaired: 0,
    visual_impaired: 0,
    clean_effects: 0,
    attached_pic: 0,
    timed_thumbnails: 0
  }
}
extracted:
{
  size: { width: 1920, height: 1080 },
  bitRate: 447244,
  duration: 30033,
  creationDate: 1672570800000,
  fileSize: 2247292,
  fps: 30
}
------------d.mp4------------------
raw data:
{
  index: 0,
  codec_name: 'h264',
  codec_long_name: 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10',
  profile: 'Baseline',
  codec_type: 'video',
  codec_time_base: '1/60',
  codec_tag_string: 'avc1',
  codec_tag: '0x31637661',
  width: 848,
  height: 480,
  coded_width: 848,
  coded_height: 480,
  has_b_frames: 0,
  sample_aspect_ratio: 'N/A',
  display_aspect_ratio: 'N/A',
  pix_fmt: 'yuv420p',
  level: 31,
  color_range: 'tv',
  color_space: 'bt709',
  color_transfer: 'bt709',
  color_primaries: 'bt709',
  chroma_location: 'left',
  field_order: 'unknown',
  timecode: 'N/A',
  refs: 1,
  is_avc: 'true',
  nal_length_size: 4,
  id: 'N/A',
  r_frame_rate: '30/1',
  avg_frame_rate: '30/1',
  time_base: '1/600',
  start_pts: 0,
  start_time: 0,
  duration_ts: 7740,
  duration: 12.9,
  bit_rate: 1637415,
  max_bit_rate: 'N/A',
  bits_per_raw_sample: 8,
  nb_frames: 387,
  nb_read_frames: 'N/A',
  nb_read_packets: 'N/A',
  side_data_type: 'Display Matrix',
  displaymatrix: '',
  rotation: '-90',
  tags: {
    rotate: '90',
    creation_time: '2023-01-01T11:00:00.000000Z',
    language: 'und'
  },
  disposition: {
    default: 1,
    dub: 0,
    original: 0,
    comment: 0,
    lyrics: 0,
    karaoke: 0,
    forced: 0,
    hearing_impaired: 0,
    visual_impaired: 0,
    clean_effects: 0,
    attached_pic: 0,
    timed_thumbnails: 0
  }
}
extracted:
{
  size: { width: 480, height: 848 },
  bitRate: 1637415,
  duration: 12900,
  creationDate: 1672570800000,
  fileSize: 2746160,
  fps: 30
}
------------c.mp4------------------
raw data:
{
  index: 0,
  codec_name: 'h264',
  codec_long_name: 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10',
  profile: 'Baseline',
  codec_type: 'video',
  codec_time_base: '1/60',
  codec_tag_string: 'avc1',
  codec_tag: '0x31637661',
  width: 848,
  height: 480,
  coded_width: 848,
  coded_height: 480,
  has_b_frames: 0,
  sample_aspect_ratio: 'N/A',
  display_aspect_ratio: 'N/A',
  pix_fmt: 'yuv420p',
  level: 31,
  color_range: 'tv',
  color_space: 'bt709',
  color_transfer: 'bt709',
  color_primaries: 'bt709',
  chroma_location: 'left',
  field_order: 'unknown',
  timecode: 'N/A',
  refs: 1,
  is_avc: 'true',
  nal_length_size: 4,
  id: 'N/A',
  r_frame_rate: '30/1',
  avg_frame_rate: '30/1',
  time_base: '1/600',
  start_pts: 0,
  start_time: 0,
  duration_ts: 7740,
  duration: 12.9,
  bit_rate: 1637415,
  max_bit_rate: 'N/A',
  bits_per_raw_sample: 8,
  nb_frames: 387,
  nb_read_frames: 'N/A',
  nb_read_packets: 'N/A',
  side_data_type: 'Display Matrix',
  displaymatrix: '',
  rotation: '-90',
  tags: {
    rotate: '90',
    creation_time: '2023-01-01T12:00:00.000000Z',
    language: 'und'
  },
  disposition: {
    default: 1,
    dub: 0,
    original: 0,
    comment: 0,
    lyrics: 0,
    karaoke: 0,
    forced: 0,
    hearing_impaired: 0,
    visual_impaired: 0,
    clean_effects: 0,
    attached_pic: 0,
    timed_thumbnails: 0
  }
}
extracted:
{
  size: { width: 480, height: 848 },
  bitRate: 1637415,
  duration: 12900,
  creationDate: 1672574400000,
  fileSize: 2746160,
  fps: 30
}

Here: https://github.com/bpatrik/pigallery2/blob/3e6bb7bff719f2c768c8ddf939cc63089a8f7776/src/backend/model/threading/MetadataLoader.ts#L20-L93

The app uses tags.creation_time to get the creation time. Based on the raw data that I get from ffprobe, it seems that the creation date does not have any time zone value.

mcwieger commented 1 year ago

Not sure what the call to action is.

Does it mean PG2 should be updated to reflect the timezone? Or to use a different timefield? Up to you of course.

Or should I update my date/times differently with Exiftool (I think with UTC is the best way, but I can be flexible).

bpatrik commented 1 year ago

I gave it some tought since my last comment.

We can decide how to understand the timezoneless date communing from ffprobe. I need to check what Date.parse does, but I assume it does in local time. I think that should be rather in UTC.

If I understand correctly that is basically what is this request about.

The othe approach is to dedust the idea of using exiftool in the background and outsource the problem:)

The problem with that the last time we tried it was extremely slowly compared to what we have. See: now https://github.com/bpatrik/pigallery2/pull/294

-- Sorry for being brief, sent from my phone.

On Fri, 4 Aug 2023, 09:52 mcwieger, @.***> wrote:

Not sure what the call to action is.

Does it mean PG2 should be updated to reflect the timezone? Or to use a different timefield? Up to you of course.

Or should I update my date/times differently with Exiftool (I think with UTC is the best way, but I can be flexible).

— Reply to this email directly, view it on GitHub https://github.com/bpatrik/pigallery2/issues/682#issuecomment-1665172896, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABZKA5VNWFXHE5HQMBOB623XTSS2ZANCNFSM6AAAAAA22OLLXI . You are receiving this because you commented.Message ID: @.***>

bpatrik commented 1 year ago

I think #710 helps with this issue. Can you check?

I rerun my little tests with some better explanation. Here it is:

------------a.mov------------------
--------------raw data------------
container: {
  major_brand: 'qt  ',
  minor_version: '512',
  compatible_brands: 'qt  ',
  creation_time: '2023-01-01T12:00:00.000000Z',
  encoder: 'Lavf57.19.100',
  date: '2023-01-01T12:00:00+0100'
} 
stream: {
  creation_time: '2023-01-01T12:00:00.000000Z',
  language: 'eng',
  handler_name: 'DataHandler',
  encoder: 'Lavc57.16.101 libx264'
}
------------app found this---------
extracted time: 2023-01-01T12:00:00.000Z (same as: 1672574400000 timestamp)
{
  size: { width: 1920, height: 1080 },
  bitRate: 588084,
  duration: 30033,
  creationDate: 1672574400000,
  fileSize: 2247292,
  fps: 30
}
------------b.mov------------------
--------------raw data------------
container: {
  major_brand: 'qt  ',
  minor_version: '512',
  compatible_brands: 'qt  ',
  creation_time: '2023-01-01T11:00:00.000000Z',
  encoder: 'Lavf57.19.100',
  date: '2023-01-01T12:00:00+0100'
}
stream: {
  creation_time: '2023-01-01T11:00:00.000000Z',
  language: 'eng',
  handler_name: 'DataHandler',
  encoder: 'Lavc57.16.101 libx264'
}
------------app found this---------
extracted time: 2023-01-01T11:00:00.000Z (same as: 1672570800000 timestamp)
{
  size: { width: 1920, height: 1080 },
  bitRate: 588084,
  duration: 30033,
  creationDate: 1672570800000,
  fileSize: 2247292,
  fps: 30
}
------------d.mp4------------------
--------------raw data------------
container: {
  major_brand: 'mp42',
  minor_version: '0',
  compatible_brands: 'mp42isom',
  creation_time: '2023-01-01T11:00:00.000000Z',
  date: '2023-01-01T11:00:00Z',
  'date-eng': '2023-01-01T11:00:00Z'
}
stream: {
  rotate: '90',
  creation_time: '2023-01-01T11:00:00.000000Z',
  language: 'und'
}
------------app found this---------
extracted time: 2023-01-01T11:00:00.000Z (same as: 1672570800000 timestamp)
{
  size: { width: 480, height: 848 },
  bitRate: 1695587,
  duration: 12900,
  creationDate: 1672570800000,
  fileSize: 2746160,
  fps: 30
}
------------c.mp4------------------
--------------raw data------------
container: {
  major_brand: 'mp42',
  minor_version: '0',
  compatible_brands: 'mp42isom',
  creation_time: '2023-01-01T11:00:00.000000Z',
  date: '2023-01-01T11:00:00Z',
  'date-eng': '2023-01-01T11:00:00Z'
}
stream: {
  rotate: '90',
  creation_time: '2023-01-01T12:00:00.000000Z',
  language: 'und'
}
------------app found this---------
extracted time: 2023-01-01T11:00:00.000Z (same as: 1672570800000 timestamp)
{
  size: { width: 480, height: 848 },
  bitRate: 1695587,
  duration: 12900,
  creationDate: 1672570800000,
  fileSize: 2746160,
  fps: 30
}

Based on this the app reads the best time it can read. At least I do not see any better creation date that it could read.

mcwieger commented 1 year ago

Appreciate the follow-up!

Not sure what/how to test. Is there a new field you're taking the date from? In that case, which field should I include in my Exiftool command so it also gets the correct UTC date?

bpatrik commented 1 year ago

I would recommend not changing the date at all as the app assumes that the date can be in any timezone and then display it in UTC. If you change the date and set the timezone to +0000, there is not much the app can do. It can only show that date as it shown in the previous post.

Yes there is a change where the date is from. Earlier it was only from the first stream. Now it's checks both the stream and the container.

-- Sorry for being brief, sent from my phone.

On Thu, 14 Sept 2023, 09:54 mcwieger, @.***> wrote:

Appreciate the follow-up!

Not sure what/how to test. Is there a new field you're taking the date from? In that case, which field should I include in my Exiftool command so it also gets the correct UTC date?

— Reply to this email directly, view it on GitHub https://github.com/bpatrik/pigallery2/issues/682#issuecomment-1718946012, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABZKA5XYX5RDFVBUF7SZKULX2KZ3TANCNFSM6AAAAAA22OLLXI . You are receiving this because you commented.Message ID: @.***>

bpatrik commented 12 months ago

I will close this due to inactivity. (a bug cleanup ahead of the next release) Feel free to reopen if you think otherwise.

gschuager commented 9 months ago

I am facing this same issue: I have both photos and videos (mp4) taken with the same device and the sorting provided by PG2 is incorrect in the same way as described above.

It seems as if PG2 is mixing date/time from photos (with time zone info) with UTC date/time from videos (interpreting these as local)

The only way I found to make the files appear in the correct order is to update the mp4 tags with exiftool without using the QuickTimeUTC=1 flag, but this leave the files with local date/time in those tags where AFAIK they are intended to contain UTC timestamps.

(BTW, thanks very much for PiGallery2)

bpatrik / pigallery2

Use of UTC for video files #682