The index page shows unaligned previews

[deflax]

Hi. Thank you for the wonderful gallery :) I've searched for a similar issue but didn't find one. Found something about my second one for pruning cache on source delete at #55 , because i also deal with video and at first i thought that's because I've deleted some files and the index caches them or something like that, but then i tried a fresh start with cleaned up the database and index.

So i think there is an edge case where my source video doesn't have its dimensions detected so it shows:

Dimensions | undefinedxundefined in the info. Moving forward and backwards once i reach the video preview works fine so i can found the video in question which i dont see well on the front page:

The black rectangular should be the video preview but the position and size are weird. I can see part of the total time calculated. Perhaps the reason is my source video has somewhat broken headers since its a stream dump, but the fallback dimensions of the preview mess up the design in this case.

Thank you for your time.

[xemle]

Hi @deflax

thank you for your possitiv feedback and your report about your issue.

The video size is extracted by exif information from database exif.js by ImageWith and ImageHeight property of media's ...-exif.json file in the storage file. Than it is overwitten by database video.js from the ...-ffprobe.json storage file.

So it seems that your video has no video stream for video.js and the fallback of exif.js breaks the layout. Would you mind to post the contents of ...-ffprobe.json and ...-exif.json from your video file? The files start with the media id in the storage directory. So I can do further investigations.

[deflax]

Thank you for the quick reply! Looks like exif.json wasnt created at all. the ffprobe json looked okay (had width and height). Found this in the debug log:

{"level":20,"time":"2024-01-03T14:14:27.049Z","pid":282,"hostname":"e0bb675da0e6","module":"extractor.apiEntry","levelName":"debug","msg":"Use api server http://archive-api:3000"}
{"level":40,"time":"2024-01-03T14:14:37.265Z","pid":282,"hostname":"e0bb675da0e6","module":"extractor.image.exif","levelName":"warn","err":{"type":"Error","message":"timeout: waited 5000ms","stack":"waited 5000ms\n    at BatchProcess._BatchProcess_onTimeout (/app/node_modules/batch-cluster/dist/BatchProcess.js:411:113)\n    at Timeout._onTimeout (/app/node_modules/batch-cluster/dist/BatchProcess.js:300:190)\n    at listOnTimeout (node:internal/timers:569:17)\n    at process.processTimers (node:internal/timers:512:7)"},"msg":"Could not extract exif of 3cc897d:vod:2023-12-31_05-14-16-330514.mp4: Error: timeout: waited 5000ms"}
{"level":20,"time":"2024-01-03T14:14:38.979Z","pid":282,"hostname":"e0bb675da0e6","module":"extractor.video.ffprobe","levelName":"debug","duration":1714,"msg":"Extracted video meta data from 3cc897d:vod:2023-12-31_05-14-16-330514.mp4"}
{"level":20,"time":"2024-01-03T14:14:40.966Z","pid":282,"hostname":"e0bb675da0e6","module":"extractor.video.frameExtractor","levelName":"debug","duration":1986,"msg":"Extracted 1 video frames from /data/vod/2023-12-31_05-14-16-330514.mp4"}

It was fine on the other two videos tho:

{"level":20,"time":"2024-01-03T14:24:22.512Z","pid":440,"hostname":"e0bb675da0e6","module":"extractor.image.exif","levelName":"debug","duration":466,"msg":"Extracted exif data from 353672e:vod:2024-01-03_14-21-12-060118.mp4"}
{"level":20,"time":"2024-01-03T14:24:37.954Z","pid":440,"hostname":"e0bb675da0e6","module":"extractor.image.exif","levelName":"debug","msg":"Close exiftool"}

its fine if you dont want to handle shady video sources but i guess if exif fails it could have a placeholder or something like that.

[xemle]

It is the first time, that I see a timeout of exiftool from the extractor

Do I understand you problem with your video file right: exiftool fails by a timeout and ffprobe fails, too? Is there any usable information from the ...-ffprobe.json storage file for it?

[deflax]

Sorry, I wasn't clear enough - ffprobe works, exif doesnt. Here is the attachment:

ffprobe.json

[xemle]

Thank you for your clarification and the ffprobe file.

I've checked the video.js file for the database creation and it seems that height and width is only taken if the video is rotated. So if exiftool fails on landscape videos the width and height are unknown. Which seems to be the case on your side.

I will fix this soon so that the with and height are always taken by ffprobe.

Further, I am curious if exiftool on the commandline fails on your video file? Or maybe it needs more time than 5sec? Since exiftool is the bullet proven swiss army knife for media files I do not expect such error which you are reporting.

[xemle]

I have released 1.14.6 with the fix by 20abc70

Please run ./gallery.js run import (depending on your installation) to rebuild your database.

[deflax]

Thank you for the hotfix. The index is working fine now.

I did some tests from inside the container using

#!/usr/bin/env node

const ExifTool = require("exiftool-vendored").ExifTool
const exiftool = new ExifTool({ taskTimeoutMillis: 5000 })

exiftool
  .read("/data/vod/2023-12-31_05-14-16-330514.mp4")
  .then((tags /*: Tags */) =>
    console.log(tags)
  )
  .catch((err) => console.error("Something terrible happened: ", err))

with taskTimeoutMillis: 5000 we have:

Something terrible happened:  Error: BatchCluster has ended, cannot enqueue -json
-struct
-fast
-*Duration*#
-GPSAltitude#
-GPSLatitude#
-GPSLongitude#
-GPSPosition#
-Orientation#
-all
/data/vod/2023-12-31_05-14-16-330514.mp4
-ignoreMinorErrors
-execute

    at BatchCluster.enqueueTask (/app/node_modules/batch-cluster/dist/BatchCluster.js:186:25)
    at f (/app/node_modules/exiftool-vendored/dist/ExifTool.js:361:38)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async g (/app/node_modules/exiftool-vendored/dist/AsyncRetry.js:8:20)

With taskTimeoutMillis: 50000 it works:

/app # ./exiftest.js 
{
  SourceFile: '/data/vod/2023-12-31_05-14-16-330514.mp4',
  errors: [],
  tz: 'UTC',
  tzSource: 'defaultVideosToUTC',
  Duration: 0,
  PreviewDuration: 0,
  SelectionDuration: 0,
  TrackDuration: 0,
  MediaDuration: 0,
  ExifToolVersion: 12.72,
  FileName: '2023-12-31_05-14-16-330514.mp4',
  Directory: '/data/vod',
  FileSize: '1161 MB',
 ...

I did some timing

real    0m 6.87s
user    0m 0.18s
sys 0m 0.04s

[xemle]

Thank you for the numbers.

So it seems that the timeout of 5s is to less for your usecase since it takes almost 7s.

For me it is unclear how to solve this issue. Your issue is the first one which reports a timeout of exiftool. One solution could be to raise the timeout statically to 10s (by const exiftool = new ExifTool({ taskTimeoutMillis: 10000 })) or to add a configuration parameter. Depending on the reasons of the timeout one or the other might be better.

Is the timeout depending on your system? So which system are you running the gallery? Is it a powerful desktop system or low powered Rasperry Pi or NAS?

Does the timeout depends on your filesize which is 1.1GB? I do not know since I do have also video files having some GB data. Maybe it depends on the content of your VOD?

Do you have an idea for other timeout reasons? Maybe there are similar issues on the exiftool project page. Would you mind to have a quick look?

[deflax]

In my case the storage is a good old magnet HDD in md-raid 1 (so basically single disk performance) which would explain it.

Now that exiftool falls back to ffprobe i consider it solved, but for sure if you have to use it for video files i guess an environment variable should be fine. Im not sure which method is preferred, as i always thought EXIF was for images only and its not guaranteed for video formats. In my case i d'ont care how much time the software will require to prepare the files for the gallery, so i could just point a higher value for this library.

Also it's a wild guess on how video formats work, but i imagine that in some cases it does require some more time to find the metadata, especially if the source for its creation was a running stream.

Anyway I did some testing with a slightly larger 2GB file:

/app # time ./testexif.js 
...
  FileName: '2024-01-05_12-36-13-255798.mp4',
  Directory: '/data/vod',
  FileSize: '1957 MB',
...
  ImageSize: '1920x1080',
...
}
^CCommand terminated by signal 2
real    0m 16.77s
user    0m 0.20s
sys 0m 0.05s
/app # time ffprobe /data/vod/2024-01-05_12-36-13-255798.mp4
  Duration: 01:20:28.83, start: 0.000000, bitrate: 3242 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080, 3095 kb/s, 30.01 fps, 
  Stream #0:1[0x2](und): Audio: mp3 (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
real    0m 2.04s
user    0m 1.06s
sys 0m 0.66s

These are the best values ive got after multiple attempts. With ffprobe the read speed goes up to 60-70MB/s which is reasonable for the magnetic drive. With exif i cant reach more than 6-7MB/s so i guess its just slower for this kind of operation.

[xemle]

Hi @deflax - I am sorry for the late reply

Thank you very much for the numbers. It seems odd that exiftool has only 6-7 MB/s while ffprobe has 60-70 MB/s. Currently I can not explain why the throughput is so different.

Stackoverflow mentions some time differences due piping the file but I do not think it is the same here. And I stumbled across a large file support issue for exiftool but do not know if that would fix the speed... If you have time, you can check your testexif.js test script with ExifToolOptions:

...
const exiftool = new ExifTool({ taskTimeoutMillis: 5000, exiftoolArgs: ['-api', 'largefilesupport=1'] })
...

If this does not help the timeout needs to be customizable...

[deflax]

Hello. I tried what you've recommended but exiftool just turned into a zombie, so I couldn't test it. Here are my three attempts:

 475458 pts/0    Z      0:06  |   \_ [exiftool] <defunct>
 475557 pts/0    Z      0:05  |   \_ [exiftool] <defunct>
 475653 pts/0    Z      0:06  |   \_ [exiftool] <defunct>

On execution the script didnt return anyting it just exited quietly, but later ive checked the process list.

I think the approach here should be to make ffprobe the priority service for video files and if it fails it should use exiftool as fallback (with configurable timeouts of course)

[xemle]

I tried what you've recommended but exiftool just turned into a zombie, so I couldn't test it.

Thank you for testing it. It seems that -api largefilesupport=1 does not solve the issue.

I think the approach here should be to make ffprobe the priority service for video files and if it fails it should use exiftool as fallback (with configurable timeouts of course)

I keep an eye on the configuration of the timeout for the exif extractor and will report here if this is implemented. Currently width and height will be used by ffprobe output if exiftool fails. So that should be fine for the moment.

Since this is a spare time project and I am working on a UI rebrush it will take some time. PRs are welcome, too. Thank you for your time, testing and feedback

[xemle]

Closing this issue due inactivity.

Please reopen it, if it becomes active again.