Show videos as well [$75]

oparoz commented 9 years ago

Showing the thumbnails is easy. A designer just needs to design a play button overlay to indicate this is a video. Integrating the 2 players (image and video) in the slideshow so that they look like one and the same is a bit harder. The real hurdle here is that most video formats don't play in a browser without using Flash. The best solution would be to transcode videos, kind of like what we do with images, we create a thumbnail which can be viewed by the browser.

I definitely think this should be optional.

If you want to read more comments: https://github.com/owncloud/apps/issues/1437

Caveats / Risks

It's not possible to play videos located on remote shares. We need to be able to scan the file's header to be able to determine if we can play it
Phase 1
Analyse files with video media types in order to be able to only pick the ones we can support via an HTML5 video player. This requires some external dependencies, per example: https://github.com/JamesHeinrich/getID3
Make video support optional because of the required dependencies
Phase 2
Add supported files to the list of files located in the scanned album
Add a play button overlay
Pick an HTML5 player
Add the player to the slideshow
Phase 3
Style the player
Allow seeking (only works if core supports it)
Phase 4 (out of scope)
Transcode formats

There is a $75 open bounty on this issue. Add to the bounty at Bountysource.

jancborchardt commented 9 years ago

For the play button overlap, you can simply use a div with .icon-view-play with some darker half-transparent background. :) I will do detail styling.

jancborchardt commented 9 years ago

cc @karlitschek @DeepDiver1975 FYI

karlitschek commented 9 years ago

nice

jancborchardt commented 9 years ago

cc @VicDeo of our current videoviewer app on this, as well as @Instagit of this awesome new video player app: https://github.com/Instagit/videojs-sublime

oparoz commented 9 years ago

The problems rarely come from the players, but from the fact that we usually have no idea which files will play or leave users hanging... So in terms of usability, phase 1 is crucial, but requires external dependencies like mplayer.

jancborchardt commented 9 years ago

@oparoz well the player of @Instagit has a simple message that the file type is not supported. I’d say that’s fine for the first pass. Because the video is shown and you don’t wonder where it is, and you are informed why it can not be played.

oparoz commented 9 years ago

I don't think it's fine in the Files app as the user will have a hit and miss experience. Videos with thumbnails might not play. That's very confusing. Videos without thumbnails (probably rare) might never be clicked on.

It's worse in Gallery as we would be showing thumbnails of unplayable media, leading to frustration.

I've been there. Installed video players, uploaded a mixed collection and couldn't play most of it. The players only support very specific formats.

jancborchardt commented 9 years ago

Ok – but the situation would not be any different compared to now. So changing the player to a better looking and working one (on mobile as well) would be a great improvement already.

The mplayer stuff and extra dependencies is a further enhancement, but a slightly different tangent.

oparoz commented 9 years ago

Yes, but that has nothing to do with Gallery. People install an app and get a hit and miss experience. After that, they may decide to remove the app.

If Gallery claims to support videos and only offers patchy support, we're going to get lots of requests from people.

devurandom commented 9 years ago

If said player (@jancborchardt's recommendation) can display a message, won't ownCloud/Gallery also be able to determine whether the file can be played or not? And then display the thumbnail/preview greyed out, with some explanatory text in front of it?

It has to show all possible thumbnails anyway, because it's IMO supposed to show me all files of a certain type (i.e. videos and pictures), not just the ones it can play (otherwise I'd wonder why Gallery completely hides a certain file - without ever knowing the reason). So the server generates the list as usual, and Gallery just desaturises the preview/thumbnail using some JS script on the client, and maybe adds a text somewhere (for large previews that can well be the preview image itself).

Instagit commented 9 years ago

The best you can do without starting the video is try to guess if the file can be played with canPlayType(): http://www.leanbackplayer.com/test/h5mt.html

With only the file extension, there's no way to say for sure if the file can be played, because that depends on the codecs used.

With external tools that detect the codecs like mplayer mentioned here or FFmpeg, the guess could come close to 100% precision, but I think that would be a lot of work to implement and could also slow down loading times when there are many videos in the gallery.

oparoz commented 9 years ago

With only the file extension, there's no way to say for sure if the file can be played, because that depends on the codecs used.

Exactly and that's the source of a lot of frustration and misunderstanding because people will say: "I have this mkv or MP4 and the player gives me nothing/an empty video/no audio/etc."

but I think that would be a lot of work to implement

I don't think so, but it does require to use unsafe PHP methods.

and could also slow down loading times when there are many videos in the gallery.

Not if we store the format the first time we encounter a new video. It's similar to the current effort to support EXIF meta data extracted from images.

If said player can display a message, won't ownCloud/Gallery also be able to determine whether the file can be played or not?

Yes, but only after clicking on the video and as mentioned earlier, you may get a thumbnail, but the player won't show you the video. Then it's possible to grey-out the video, but the app will forget about it and you'll click again the next time you launch the app.

It has to show all possible thumbnails anyway, because it's IMO supposed to show me all files of a certain type (i.e. videos and pictures), not just the ones it can play (otherwise I'd wonder why Gallery completely hides a certain file - without ever knowing the reason)

I agree. Gallery displays broken JPEGs so that the user can download them. We could do the same with videos, showing the video icon instead of a thumbnail and letting the user try his luck. If we ever implement a way to detect the codec, then we can enable thumbnail previews.

oparoz commented 9 years ago

@Instagit - Does you player detect the VLC plugin and use it?

oparoz commented 9 years ago

Found it: https://github.com/Afterster/videojs-vlc

devurandom commented 9 years ago

If said player can display a message, won't ownCloud/Gallery also be able to determine whether the file can be played or not?

Yes, but only after clicking on the video and as mentioned earlier, you may get a thumbnail, but the player won't show you the video. Then it's possible to grey-out the video, but the app will forget about it and you'll click again the next time you launch the app.

You could store the result of that examination, as long as the player is able to give you a list of codecs used in the file, or similar.

I assume one should not generally store "playable: yes/no" in the database anyway, as each browser might support a different list of codecs.

oparoz commented 9 years ago

Yes, if the player gives an accurate list of codecs contained in the file, but I don't think it can.

oparoz commented 9 years ago

The thing to remember is that showing video online is hard if you don't pre-process uploaded files to offer various compatible formats. That's the reason I'm not keen on "just" adding a video player in Gallery as it will offer a poor experience.

If we can test for the presence of the VLC plugin, then we could offer a decent solution as the decoder would be outside the browser, but that only works on desktop afaik.

oparoz commented 9 years ago

This is the proper way to offer video, on all platforms: http://www.mediafire.com/developers/multimedia_api/video/

They're probably using a commercial video encoder though and that would explain the choice of FLV for the output, but the workflow is the same:

Detect and store if transcodable
Prepare stream
Use stream

oparoz commented 8 years ago

Someone has tested the viability of trying to use the VLC plugin to support more formats, but the UX is poor unfortunately due to several limitations linked to having to use a legacy browser extension: https://apps.owncloud.com/content/show.php/Video+Viewer+Plus?content=174666

A drawback is that VLC only supports HTTP basic authentication. Therefore, the server must be configured to allow this and the login credentials must be (re)submitted with each video view. As NPAPI support is considered legacy by most modern browsers, I'm not sure whether I'll put more energy into this...? However, using the browsers formdata save- and autofill-routines will help to ease the login process.

jancborchardt commented 8 years ago

By the way @oparoz, we now have the awesome https://github.com/owncloud/files_videoplayer by @Instagit in core for 9.0.

First step for Gallery integration could be to just show videos in the overview (with a »play icon« overlay to signify it’s a video) and on click, instead of opening the picture detail view, open the files_videoplayer.

oparoz commented 8 years ago

For me the first step remains the same. All videos have to be transcoded, just like all images have previews. Otherwise we'll just get tons of reports just like this one: https://github.com/owncloud/files_videoplayer/issues/17

jancborchardt commented 8 years ago

Really, all of them? Is that necessary for previews or for playing, or both?

Because neither is a hard blocker for video support. First iteration could be video icon + title and clicking will open the videoplayer overlay.

oparoz commented 8 years ago

Really, all of them? Is that necessary for previews or for playing, or both?

It's necessary to at least parse all videos. The engine is smart enough to know if transcoding should take place and a flag in the fileinfo array would indicate if the preview should be used instead of the original. It's the only way to offer a good UX.

Because neither is a hard blocker for video support. First iteration could be video icon + title and clicking will open the videoplayer overlay.

We've discussed it earlier in this issue. That kind of works for Files where it doesn't matter that much that half the videos don't play, but would be really bad in Gallery.

We could let users decide by having an opt-in switch in gallery.cnf

Instagit commented 8 years ago

With a good server setup, real time transcoding would be the most elegant way to solve this, but it depends very much on the hardware. Many servers have low power CPUs and real time transcoding is not an option, because they won't be able to encode with 25 FPS, even with 100% CPU usage. ownCloud can't even use 100% CPU for this though, because it will make the rest of the server unresponsive.

Do you have any hardware targets for ownCloud or statistics about the users' hardware?

oparoz commented 8 years ago

Do you have any hardware targets for ownCloud or statistics about the users' hardware?

@Instagit - I don't have access to any hardware stats, but packages such as the oC-WD pi-drive will be a good testing starting point.

The main problem remains though, we need to be able to identify what can be played prior to building the photowall. Transcoding in advance solves that problem, but will take hours, even days and lots of storage. The main advantage is that the process is only done once. There could be an app which tests the setup and generates some parameters which could be used to transcode in real-time, but the streaming experience will depend on the source material.

jancborchardt commented 8 years ago

What's your take here @Deepdiver1975 @karlitschek?

karlitschek commented 8 years ago

Hmm. I personally would try to avoid transcoding. Even with half decent hardware it is a huge load on the server. Also getting the user experience right is hard. Why doens't my video show up immediately after uploading? Why is the fan of the device constantly on? Why is ownCloud suddenly so slow.... I personally would try to avoid transcoding. We should try to generate thumbnails if possible. If not we just show a placeholder. And we should try to play what is supported by the browser and ignore the rest. (As the videoplayer app already does)

Users get the esoteric video formats don't work.

oparoz commented 8 years ago

If I'm referring to the original plan in the OP. Transcoding is the last step, a nice thing to have. So, phase 1 is still to build a video parser which outputs either the full video specs or a flag to let apps know that that particular video can be played via an HTML5 player.

karlitschek commented 8 years ago

A lot of people run ownCloud on very slow hardware like ARM boards or NAS boxes. video transcoding is a very cpu heavy task. I don't think this is something that should be done by default or in the background. Maybe if a user clicks on a file on demand because then they hopefully understand that their hardware is too weak to do this. If we start this in the background without the knowledge of the users then this basically means that we drop support for weak hardware. Not good.

oparoz commented 8 years ago

If we start this in the background without the knowledge of the users then this basically means that we drop support for weak hardware. Not good.

At the same time, it means anybody with decent hardware or who is willing to sacrifice one core for processing will be penalized by not being able to watch their videos without uploading them to a 3rd party service or without pre-processing them via a set of external tools. That last point could be documented as part of phase 1. That way people would know that only 2-3 types of video formats are supported and convert their collection if it's important to them to be able to play these files from within ownCloud.

oparoz commented 8 years ago

This issue was moved to nextcloud/gallery#62

serenity-networks commented 8 years ago

Is there any way either the java or vlc videojs plugins could be enabled in ownCloud (by the server admin)? It seems that between these two plugins any video format could be supported. I understand they are natively supported by the browser. Am I missing something here?

videojs-java: Allows you to play AIFF, AVI, GSM, MID, MPG, MP2, MOV, AU and WAV medias through Java within Video.js.

videojs-vlc: Allows you to play all media through VLC web plug-in within Video.js.

The VLC plugin is a fallback plugin and it's insanely easy to enable if you know where to enable it.

Video.js VLC

Video.js VLC Tech plug-in

A Video.js tech plugin that add VLC Media Player fallback.

Getting Started

VLC Web Plugin must be installed on the browser to work.

Once you've added the plugin script to your page, you can use it with any supported video:

Include JavaScript files

<script src="video.js"></script>
<script src="videojs-vlc.js"></script>

And add this new tech to the player:

data-setup='{ "techOrder": ["vlc"] }'

Video.js Java

Video.js Java Tech plug-in

A Video.js tech plugin that add AIFF, AVI, GSM, MID, MPG, MP2, MOV, AU and WAV stream support through Java.

Getting Started

Once you've added the plugin script to your page, you can use it with any supported video:

Include JavaScript files

<script src="video.js"></script>
<script src="videojs-java.js"></script>

And add this new tech to the player:

data-setup='{ "techOrder": ["java"] }

There's also a working example of the plugin you can check out if you're having trouble.

Plugin Options

This plugin has a global configuration to setup JAR file location.


<script>
    videojs.options.java.jar = "video-js.jar";
</script>

All info taken from the videojs site, here: https://github.com/videojs/video.js/wiki/Plugins

serenity-networks commented 8 years ago

On another note, would there be an equally easy way for someone to enable the Projekktor plugin?? It supports flash and VLC fallback.

As it stands right now video's recorded on an iphone are not playable on ownCloud. It would be very helpful to at least have that.

http://www.projekktor.com/

Instagit commented 8 years ago

@serenity-networks

ownCloud should play all videos the browser can play, so if the iPhone records videos that can't be played in the browser, that's pretty much Apple's fault. :)

On the other points, I think we have decided against video support via plugins in the past, which I think was a good call. Flash fallback for Video.js was disabled for that reason.

jancborchardt commented 8 years ago

@serenity-networks as @oparoz mentioned above, active development moved to https://github.com/nextcloud/gallery/ :) So we recommend to upgrade

DeepDiver1975 commented 8 years ago

Let's see what next version of owncloud will bring ...

serenity-networks commented 8 years ago

I look forward to the next version. @jancborchardt From what I'm reading on the NextCloud gallery git I'm not seeing anything related to video. Am I missing something? I'm currently running the latest and greatest NextCloud, and loving it!

The supported media types on the git you linked doesn't seem to mention video.

  'preview_max_scale_factor' => 1,
  'enabledPreviewProviders' =>
  array (
    0 => 'OC\\Preview\\PNG',
    1 => 'OC\\Preview\\JPEG',
    2 => 'OC\\Preview\\GIF',
    11 => 'OC\\Preview\\Illustrator',
    12 => 'OC\\Preview\\Postscript',
    13 => 'OC\\Preview\\Photoshop',
    14 => 'OC\\Preview\\TIFF'
  ),

Thanks!

owncloud / gallery