search

nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
27.18k stars 4.04k forks source link

Preview of MP4 Videos failing to generate with ffmpeg/avconf if larger than 5MB #1475

Closed sascha-schieferdecker closed 8 years ago

sascha-schieferdecker commented 8 years ago

Steps to reproduce

  1. Upload MP4 video larger than 5MB, with the moov atom at the end of the file, e.g. from an older android phone. About the moov atom: https://www.adobe.com/devnet/video/articles/mp4_movie_atom.html

    Expected behaviour

Video Thumbnail should be generated, works with MPG files.

Actual behaviour

avconf/ffmpeg fail because only the forst 5MB of the video are taken into account, see https://github.com/nextcloud/server/blob/master/lib/private/Preview/Movie.php#L57

If the moov atom is at the beginning of a video, the preview generation works. If the the moov atom is at end of the video, the generation fails. If it is a local file would it be possible to load it completely? Removing the limit in the stream_get_contents call generates the preview just fine. Or maybe would it be possible to streamline the video using something like qt-faststart first?

Server configuration

Operating system: Debian 8

Web server: Apache 2.4

Database: MySQL

PHP version: PHP 7.0.11

Nextcloud version: Nextcloud 10.0 (stable)

Updated from an older Nextcloud/ownCloud or fresh install: No

Where did you install Nextcloud from: Homepage

Please forgive me if I don't post any other configuration details, IMHO they are not relevant to described behaviour. ``

nickvergessen commented 8 years ago

Hmm, disabling the switch based on storage sounds a bit weird to me. I'd say this is fine as is, if the file is too big and does not have "progressive download," "fast start," "use streaming mode," or similar options, I guess that is simply what you end up with?

sascha-schieferdecker commented 8 years ago

Okay, if you define it as expected behaviour, this is what a user ends up with. I am a first time user of nextcloud (never used owncloud). I wondered why there was no preview of most mp4 videos, so I dug into the code. I think I'm not the only user wondering if this behaviour is an error.

In my case I converted the videos to fast start and my problem is solved. But I think most users expect a working preview generation.

Maybe this is more of feature request, depending on how you see it.

binarydad commented 4 years ago

I'm noticing this now, but they're for SMB shares on my local network. Is there a way to force preview generation for videos over 5MB?

sascha-schieferdecker commented 4 years ago

AFAIK no, but you could convert the video with ffmpeg to "fast start", as described here.

Or take a look at qt-faststart, which moves the required information to the beginning of the file.

binarydad commented 4 years ago

Thanks, but how can I tell nextcloud to use these options when generating previews?

sascha-schieferdecker commented 4 years ago

You can't AFAIK. Just run your videos through qt-faststart or ffmpeg in a batch. That's how I solved it.

binarydad commented 4 years ago

Alright, thank you for that. I'll take a look.

Really bizarre... 5MB is nothing. This issue was racking my brain for months, as video previews for larger movies worked fine on local storage, but failed over SMB. At least I know why now...

kaipee commented 4 years ago

Is there a fix scheduled for this? I'm still failing to generate any video (MP4) thumbnails for files stored on 1Gbps SMB share

liranbx commented 3 years ago

I'm also a first time user running into this thing and had to dive into the code to figure it out (too bad this issue was not listed on my google searches). Was there any progress on this? Is there a branch you're working on?

binarydad commented 3 years ago

So, the way I "solved" this was just bypass the code that is distinguishing between a local and a remote file.

In /var/www/nextcloud/lib/private/Preview/Movie.php, in the getThumbnail() method, is the line $absPath = $this->getLocalFile($file, 5242880); // only use the first 5MB. This is where it's loading a file, but only capping it to 5MB. The getLocalFile() method is inside ProviderV2.php (same directory), which determines whether the file is local or not, eventually setting the $useTempFile boolean flag. If this is true, this is where it loads in an external file, but only 5MB. However, if it's told to load a local file, it will do so, and I've noticed this works fine for SMB files as well.

My fix was to bypass the $useTempFile flag and force getLocalFile() to treat the "external" (SMB shared) file also as local, which allows FFMPEG to read the full file and generate a thumbnail.

I've also tried simply increasing the max file size from 5MB to something much higher, but I feel the overhead of loading an external file the way it does was slower than simply bypassing it and treating it as a local file. I don't know much about PHP but I imagine going directly to the file system to get the file rather than PHP loading it into memory first would be quicker. And I have noticed that thumbnails for larger videos (several GB at least) generate thumbnails a little quicker.

Here's my glamorous fix in ProviderV2.php: 🙃

image

kaipee commented 3 years ago

@binarydad Just confirming that this 'fix' works great for me too. I have confirmed MP4 large videos are now very quickly generating thumbnails in the Nextcloud Android app, when they completely failed before.

BTW your last sentence should be updated, you've called the file PreviewV2.php instead of ProviderV2.php

liranbx commented 3 years ago

Sorry for the late comment. From what I understand your solution works because you work with a high speed LAN SMB storage, sadly the same solution won't work for me as I use S3 object storage as a primary storage. It seems as there is no simple solution to creating mp4 previews on a truly remote storage except pulling the entire file without pre-processing the file, because the moov atom placement might be at the end of the file.

My thought is to create some flow application that will pre-process every video file upload to convert it either to webm format or to use qt-faststart to relocate the moov atom.

binarydad commented 3 years ago

Are you using the preview generator CRON job? You can increase the file limit from 5MB to something more reasonable for your case and schedule the preview generation to at least run at night time.

------ Original Message ------ From: "liranbx" notifications@github.com To: "nextcloud/server" server@noreply.github.com Cc: "Ryan Peters" ryan@binarydad.com; "Mention" mention@noreply.github.com Sent: 2/11/2021 1:14:51 PM Subject: Re: [nextcloud/server] Preview of MP4 Videos failing to generate with ffmpeg/avconf if larger than 5MB (#1475)

Sorry for the late comment. From what I understand your solution works because you work with a high speed LAN SMB storage, sadly the same solution won't work for me as I use S3 object storage as a primary storage. It seems as there is no simple solution to creating mp4 previews on a truly remote storage except pulling the entire file without pre-processing the file, because the moov atom placement might be at the end of the file.

My thought is to create some flow application that will pre-process every video file upload to convert it either to webm format or to use qt-faststart to relocate the moov atom.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nextcloud/server/issues/1475#issuecomment-777689433, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANN2XYEV6R5CVGUVAGXH4DS6QNBXANCNFSM4CQI4RQA.

liranbx commented 3 years ago

Problem is that with the large size of video file it might result in a very high bill to the S3 provider. Additionally, it seems that many of my videos have their moov atom at the very end of the file so 20M will be just enough for a 20M video. Given today's resolutions it doesn't cover much (changing the cap to 10M was sufficient for a 3 seconds video).

utack commented 3 years ago

Uhm, why is this closed?
The problem still seems to exist

patricktotzke commented 3 years ago

I agree this limit should at least be configurable. I've mounted a local directory as external file source so that's as fast as it gets yet the default settings don't generates thumbnails for videos produced by my mobile phone. Running an external conversation command on all these files. looks cumbersome..

bkalbfus commented 2 years ago

Please reopen this issue. I've been living with it for a while now. My data is on an external USB drive. Using NextCloudPi and it's on Nextcloud 22.2.2.0 at this point. Thanks!

br4yd commented 9 months ago

Hello, experiencing the same issue on Nextcloud 28.0.1 using Nextcloud AIO... Videos uploaded from an iPhone are playable without any issues. For videos uploaded from an Samsung S21 FE, a Google Pixel 2 XL or a Google Pixel 8 Pro (GrapheneOS) I get a preview image in the overview and I get sound when playing but I get no video when playing. I do get errors in the logs and that's it.

Not sure why this issue is still closed and not reopened when this issue is still existent after such a long time. Is there any known way to fix this in the current version using AIO? Unfortunately makes it impossible for me to use Nextcloud for photo and video backup when I'm not able to watch my videos.

Thanks in advance!

llucax commented 9 months ago

Hello, experiencing the same issue on Nextcloud 28.0.1 using Nextcloud AIO... Videos uploaded from an iPhone are playable without any issues. For videos uploaded from an Samsung S21 FE, a Google Pixel 2 XL or a Google Pixel 8 Pro (GrapheneOS) I get a preview image in the overview and I get sound when playing but I get no video when playing. I do get errors in the logs and that's it.

Not sure why this issue is still closed and not reopened when this issue is still existent after such a long time. Is there any known way to fix this in the current version using AIO? Unfortunately makes it impossible for me to use Nextcloud for photo and video backup when I'm not able to watch my videos.

Thanks in advance!

Your issue with not being able to play video from pixel phones is likely due to pixel phones use the x265 video codec, which is unfortunately not supported by most major browsers. There is an option to use h264 instead in the phone, but videos will take about twice as much space. The solution is to enable transcoding, for example the Memories app has the option to transcode, but it requires some extra setup and a decent CPU.

br4yd commented 9 months ago

Your issue with not being able to play video from pixel phones is likely due to pixel phones use the x265 video codec, which is unfortunately not supported by most major browsers. There is an option to use h264 instead in the phone, but videos will take about twice as much space. The solution is to enable transcoding, for example the Memories app has the option to transcode, but it requires some extra setup and a decent CPU.

Hi, yes I do indeed have this set up and still experience the issue. I opened a related issue with more details summarized by several unanswered threads around this topic including this one. -> See here https://github.com/nextcloud/server/issues/42545