No thumbnails on videos with ffmpeg

[gdureuil]

Detailed description of the problem

No previews are generated on videos, even with ffmpeg installed and configured in Lychee. Thumbnails were generated file in older revisions. Up to date with master branch.

Steps to reproduce the issue

Upload a video in album, no thumbnail. The video can't be played too.

I think composer dependencies are broken for ffmpeg-php. Here is what happens when checking with composer.

sudo -u www-data composer require php-ffmpeg/php-ffmpeg
Info from https://repo.packagist.org: #StandWithUkraine
Using version ^1.0 for php-ffmpeg/php-ffmpeg
./composer.json has been updated
Running composer update php-ffmpeg/php-ffmpeg
Loading composer repositories with package information                                      Updating dependencies
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Root composer.json requires php-ffmpeg/php-ffmpeg ^1.0, found php-ffmpeg/php-ffmpeg[v1.0.0, v1.0.1, v1.x-dev] but these were not loaded, likely because it conflicts with another require.
  Problem 2
    - lychee-org/php-exif is locked to version dev-master and an update of this package was not requested.
    - lychee-org/php-exif dev-master requires php-ffmpeg/php-ffmpeg ^0.17.0 -> found php-ffmpeg/php-ffmpeg[v0.17.0] but it conflicts with your root composer.json require (^1.0).

Use the option --with-all-dependencies (-W) to allow upgrades, downgrades and removals for packages currently locked to specific versions.
You can also try re-running composer require with an explicit version constraint, e.g. "composer require php-ffmpeg/php-ffmpeg:*" to figure out if any version is installable, or "composer require php-ffmpeg/php-ffmpeg:^2.1" if you know which you need.

Installation failed, reverting ./composer.json and ./composer.lock to their original content.

After some tests, logs from Lychee

 2022-04-11 21:57:22 -- error   -- App\Actions\Photo\Strategies\AddStandaloneStrategy::do -- 88 -- Failed to generate size variants, error was Media file is reported to be a video, but aperture (aka duration) has not been extracted
 2022-04-11 21:57:22 -- error   -- App\Image\SizeVariantDefaultFactory::extractReferenceImage -- 60 -- Media file is reported to be a video, but aperture (aka duration) has not been extracted
 2022-04-11 21:57:20 -- notice  -- App\Metadata\Extractor::extract -- 160 -- Falling back to native adapter.

Output of the diagnostics [REQUIRED]

    Diagnostics
    -------
    Warning: Dropbox import not working. dropbox_key is empty.
    Warning: lossless_optimization set to 1 but cwebp not found!
    Warning: lossless_optimization set to 1 but svgo not found!
    Warning: You may experience problems when uploading a photo of large size or handling many/large albums. Take a look in the FAQ for details.
    Warning: Default timezone not properly set; you might experience strange results when importing photos without explicit EXIF timezone

    System Information
    --------------
    Lychee Version (git):            master (d8279c8) - Data not in Cache
    DB Version:                      4.5.0

    composer install:                dev
    APP_ENV:                         local
    APP_DEBUG:                       false

    System:                          Linux
    PHP Version:                     8.1
    PHP User agent:                  Lychee/4 (https://lycheeorg.github.io/)
    Timezone:                        UTC
    Max uploaded file size:          100M
    Max post size:                   100M
    Max execution time:              90
    MySQL Version:                   10.3.34-MariaDB-0+deb10u1-log

    Imagick:                         1
    Imagick Active:                  1
    Imagick Version:                 1690
    GD Version:                      2.2.5

    Config Information
    --------------
    version:                         040500
    check_for_updates:               0
    sorting_Photos_col:              title
    sorting_Photos_order:            ASC
    sorting_Albums_col:              title
    sorting_Albums_order:            ASC
    imagick:                         1
    skip_duplicates:                 1
    small_max_width:                 0
    small_max_height:                360
    medium_max_width:                1920
    medium_max_height:               1080
    lang:                            fr
    layout:                          0
    image_overlay_type:              none
    default_license:                 none
    compression_quality:             90
    full_photo:                      1
    delete_imported:                 0
    Mod_Frame:                       1
    Mod_Frame_refresh:               30
    thumb_2x:                        1
    small_2x:                        1
    medium_2x:                       1
    landing_page_enable:             0
    landing_owner:                   
    landing_title:                   
    landing_subtitle:                
    landing_facebook:                
    landing_flickr:                  
    landing_twitter:                 
    landing_instagram:               
    landing_youtube:                 
    landing_background:              
    site_title:                      BakaGamer - Screenshots
    site_copyright_enable:           0
    site_copyright_begin:            2010
    site_copyright_end:              2022
    additional_footer_text:          
    display_social_in_gallery:       0
    public_search:                   0
    SL_enable:                       0
    SL_for_admin:                    0
    public_recent:                   0
    recent_age:                      1
    public_starred:                  0
    downloadable:                    0
    photos_wraparound:               0
    map_display:                     0
    zip64:                           1
    map_display_public:              0
    map_provider:                    Wikimedia
    force_32bit_ids:                 0
    map_include_subalbums:           0
    update_check_every_days:         3
    has_exiftool:                    0
    share_button_visible:            0
    import_via_symlink:              0
    has_ffmpeg:                      1
    location_decoding:               0
    location_decoding_timeout:       30
    location_show:                   0
    location_show_public:            0
    rss_enable:                      0
    rss_recent_days:                 7
    rss_max_items:                   100
    prefer_available_xmp_metadata:   0
    editor_enabled:                  1
    lossless_optimization:           1
    swipe_tolerance_x:               150
    swipe_tolerance_y:               250
    local_takestamp_video_formats:   .avi|.mov
    log_max_num_line:                1000
    unlock_password_photos_with_url_param: 0
    nsfw_visible:                    1
    nsfw_blur:                       0
    nsfw_warning:                    1
    nsfw_warning_admin:              0
    map_display_direction:           1
    album_subtitle_type:             oldstyle
    upload_processing_limit:         2
    public_photos_hidden:            1
    new_photos_notification:         0
    legacy_id_redirection:           1

[kamil4]

This is weird, in multiple ways.

Problems with thumbnail generation are not uncommon; it's a fragile process due to external dependencies. However, any problems with ffmpeg should not prevent videos from being played back, because the playback is done entirely by your web browser (Lychee uses HTML5 <video> tags). Are you sure that the video files you are uploading are playable by the browser though? The list of browser-supported formats is rather short; in particular h.265/hevc video codec is typically not supported. You can verify that a particular file is supported simply by loading it into your web browser from a local disk (file:///).

Now to your problems with php-ffmpeg. Lychee and its dependencies currently require version 0.17.0 and there's nothing wrong with that. From the console output you supplied, it looks like you triggered dependency conflicts by requesting a version of php-ffmpeg that's not supported by Lychee. So before we can make any further progress on this, we need to make sure that your Lychee installation is consistent. While in the top-level Lychee directory, please run the following:

$ git status

Does it report that the files composer.json or composer.lock are modified? If so, please restore their versions from the git repo:

$ git checkout -- composer.json
$ git checkout -- composer.lock

After that, to be honest, I would recommend simply deleting the whole vendor subdirectory and then running:

$ composer install

This will refetch all the external dependencies. It's probably more extreme than strictly necessary, but at least you'll be sure that all the dependencies are in order. So once you do that, try uploading your video file again and if it still doesn't work, we can investigate further.

[d7415]

From the console output you supplied, it looks like you triggered dependency conflicts by requesting a version of php-ffmpeg that's not supported by Lychee

I didn't spot it until reading your reply, but the issue was right at the start of the "checking" that @gdureuil posted:

sudo -u www-data composer require php-ffmpeg/php-ffmpeg
Info from https://repo.packagist.org: #StandWithUkraine
Using version ^1.0 for php-ffmpeg/php-ffmpeg
./composer.json has been updated

So we at least know where it came from.

[gdureuil]

Ok here are my tests

git status

sudo -u www-data git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

So I proceeded to remove my existing vendor folder, and did this.

sudo -u www-data composer install

A bunch of libraries were redownloaded.

Then tried to upload a video file again. Video is mp4 file, H264 - MPEG-4 AVC (part 10) (avc1) - Duration 30 seconds - 1920x1080 60fps.

The same problem happens. Video is uploaded, no preview, and unplayable though lychee in browser. Using the file URL, browser reads the video OK (with "View original").

To be sure, I tested with a smaller video. mp4 file, H264 - MPEG-4 AVC (part 10) (avc1) - Duration 30 seconds - 1280x720 30fps Result is still no thumbnail, but now the video is playable through lychee.

I made a public test albums with the two videos, They are downloadable too : https://screenshots.baka-gamer.net/#TFh1kSvBjcARN6OfW-ru33Qq

EDIT : The unplayable video has no frame in the abum, but you can hover it. Always does that when reuploading.

I can provide more infos if you need.

EDIT2 : Also verified that ffmpeg is still installed correcly. Debian 10 from official repositories.

ffmpeg -version
ffmpeg version 4.1.8-0+deb10u1 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 8 (Debian 8.3.0-6)

[kamil4]

Thank you for a link to an album -- that helps a lot! The second video file (the one that shows without a frame) appears to be an instance of a bug we already fixed in the past, in #859. But it looks like it's broken again (I can reproduce it) -- we'll need to investigate what's going on.

The first video file works just fine on my Lychee instance. Can you delete it, upload it again, and report any messages found in the logs?

[gdureuil]

I deleted the smaller video, and reuploaded. Still no thumbnail.

Here is the Lychee log for this upload.

2022-04-12 20:44:33 -- error   -- App\Actions\Photo\Strategies\AddStandaloneStrategy::do -- 88 -- Failed to generate size variants, error was Class "App\Image\ImageOptimizer" not found
2022-04-12 20:44:31 -- notice  -- App\Image\SizeVariantDefaultFactory::createTmpPathForReference -- 182 -- Saving JPG of raw/video file to /tmp/lycheeBLVmHG

EDIT : Also checked in database, in "photos" table. Small video has "type" as "video/mp4" Big video has "type" as "application/octet-stream", just as the old bug you linked.

[nagmat84]

On top of \App\Image\SizeVariantDefaultFactory the following line is missing

use Spatie\LaravelImageOptimizer\Facades\ImageOptimizer;

Sorry, I cannot commit a fix myself right now.

[kamil4]

Yeah, I know, I saw application/octet-stream in the info sidebar...

As to your smaller video file, it looks like enabling lossless_optimization triggers another bug, so if you disable it for the time being, you should be good to go.

[gdureuil]

I put lossless_optimization to 0 and reuploaded both videos. Small video has thumbnail and is working great. No change for big video.

[nagmat84]

BTW @kamil4: Do you know why the setting is called "lossless optimization"? I had a look into what the Spatie Image Optimizer does under the hood. It only optimizes space by reducing JPEG compression to 85%. Similar things happen for other image formats. This is anything but lossless.

[kamil4]

@nagmat84 I have no insight. This was merged when I was on a break from Lychee development during the first Covid lockdown. It's PR #659.

[kamil4]

@nagmat84 Back to the original issue, I'm guessing this broke with #1203.

PHPExif has the following code:

https://github.com/LycheeOrg/php-exif/blob/2f74ba7d7a66bc8c9af4e8e8731797d9d4103437/lib/PHPExif/Adapter/FFprobe.php#L93-L108

This doesn't work anymore, because $file does not have an extension (e.g., /tmp/phpnD8pdb). The above code is invoked from the metadata extractor, which starts with:

https://github.com/LycheeOrg/Lychee/blob/4d1b5f0df078bd6160f7815f61d1e9f0447ef6b6/app/Metadata/Extractor.php#L109-L121

Already here $fullPath does not have an extension so I'm guessing the raw-handling code is broken as well...

Any thoughts? It seems to me that we should be creating the temporary file with the right extension, or we need to pass the original file name around to a lot more places that, as it turns out, may need it...

[nagmat84]

seems to me that we should be creating the temporary file with the right extension

I'm fine with that. As it turns out that I am responsible for the break, do you want to fix it or shall I do it?

[kamil4]

I'm fine with that. As it turns out that I am responsible for the break, do you want to fix it or shall I do it?

Could you do it? I looked at the source and I'm honestly not sure how to fix it correctly. Especially since I remember that the direct motivation for that refactoring was an issue report where a user was experiencing problems with file renaming...

[kamil4]

Reopening; so far only a partial fix is in (and not for the original problem).

[nagmat84]

Could you do it? [...] Especially since I remember that the direct motivation for that refactoring was an issue report where a user was experiencing problems with file renaming...

No problem, I will see what I can do during the extended Easter weekend.

[nagmat84]

@gdureuil It seems as if we have a (temporary) fix for the problem. Before we merge the fix and this issue will be closed, I would like to ask you if we could use your video in the future as a litmus test to check if we break things again. Would this be OK with you? I would also like to add the video to our automated test suite.

[gdureuil]

Hello. No problem, you can use it for your internal tests, no worries :)

[gdureuil]

Just updated to master with "git pull" seeing you closed the issue. I uploaded videos and now everything seems to work fine for thumbnails.

Thank you very much for you help and your time on this matter !