[bug] Blank Open Graph image for posts with video attachment

amiskelly commented 3 weeks ago

Describe the bug with a clear and concise description of what the bug is.

Posts which have a video (MP4/H.264 here) attachment (e.g. https://posts.anero.id/@andrewmiskelly/statuses/01J0T7TNV06QVEXPCRSJGXHAVV) produce OG tags, which is great!

<meta property="og:image" content="https://posts.anero.id/fileserver/01FX45SE69SE30BXVYRWJYY279/attachment/small/01J0T7T0KWZGBK6YHS4XECZZ2N.jpg">
<meta property="og:image:width" content="512">
<meta property="og:image:height" content="512">

The og:image URL points to a blank, black image though (https://posts.anero.id/fileserver/01FX45SE69SE30BXVYRWJYY279/attachment/small/01J0T7T0KWZGBK6YHS4XECZZ2N.jpg).

It would be great if a proper preview image was produced, and even better if it was a little higher resolution (https://opengraph.dev/ recommends 1200 pixels wide) for display on high resolution screens. See also https://github.com/superseriousbusiness/gotosocial/issues/2776.

What's your GoToSocial Version?

0.16.0

GoToSocial Arch

arm64 binary

What happened?

The og:image URL points to a blank, black image of 512x512 pixels.

What you expected to happen?

The og:image URL points to a representative preview image of the video.

How to reproduce it?

Anything else we need to know?

No response

tsmethurst commented 3 weeks ago

This is a result of the way that GoToSocial handles video. To avoid bringing in dependencies that rely on CGO or calling out to ffmpeg, we keep video decoding to a minimum, just decoding the metadata without actually touching the video data itself (which also has benefits in terms of using less cpu time + memory).

Unfortunately this means we don't generate thumbnails for videos based on the first frame like many implementations do, we just generate a black image that's the same resolution as the video itself. To add this feature is non-trivial, since it could introduce a whole bundle of dependencies that risks compromising the portability of GtS.

I think probably a decent workaround for now would be to just skip adding opengraph preview images for media posts when the only media attached is a video 🤔 At least that would prevent just a black image from showing up, which is distracting. In future we can look for a way to resolve this properly by finding a decoder that works for our current stack and approach, but it may be a low-priority feature for a while, just so you know.

amiskelly commented 3 weeks ago

Thanks for the response, Tobi. I understand the dependency vs portability problem.

Would it be straightforward to use an og:video tag referring to the original attachment when the post is centred on a video? https://ogp.me/#type_video

Until the fediverse becomes mainstream, it's desirable to reach into silos (even Bluesky) from time to time. It seems to me that the most straightforward way of doing this is to reference a GoToSocial post and rely on its OG tags to do the rest.

daenney commented 2 weeks ago

The opengraph video tags provide metadata for a video as well as the URL the video can be retrieved at. However, it doesn't cause a client to fetch the video and generate a preview image all by itself.

For example, if you open a YouTube video and look at its opengraph tags, it still has an og:image which is what's used for the preview.

coxde commented 2 weeks ago

Perhaps it would be possible to make calling ffmpeg an option? e.g for Docker deploy, make the ffmpeg container optional. Keep the default to decode metadata only.

tsmethurst commented 2 weeks ago

We've been investigating something like this :)

superseriousbusiness / gotosocial