bluesky-social / social-app

The Bluesky Social application for Web, iOS, and Android
https://bsky.app
MIT License
13.51k stars 1.73k forks source link

[APP-932] Additional requested embed players #2314

Open pfrazee opened 11 months ago

pfrazee commented 11 months ago

We can anything that's secure and has moderation on their platform!

APP-932

ondrejholan commented 11 months ago

Sketchfab embeds would be useful for 3D artists.

seboslaw commented 11 months ago

As a side-note. The app currently automatically opts-in users to this functionality. However, from a data privacy perspective this is kind of edgy since the app now per default (at least) leaks the user's IP address to 3rd parties like Google, Meta, etc. pp. So imho this feature should be opt-in and there should be a disclaimer informing the users what information is passed onto the 3rd parties when enabling the feature.

pfrazee commented 11 months ago

Another thing to add to the list, handle the youtube mobile links that prefix with m.

pfrazee commented 11 months ago

As a side-note. The app currently automatically opts-in users to this functionality. However, from a data privacy perspective this is kind of edgy since the app now per default (at least) leaks the user's IP address to 3rd parties like Google, Meta, etc. pp.

It only contacts those services if the users tap into it and I think people generally understand that those services have to be contacted to display content from them

seboslaw commented 11 months ago

As a side-note. The app currently automatically opts-in users to this functionality. However, from a data privacy perspective this is kind of edgy since the app now per default (at least) leaks the user's IP address to 3rd parties like Google, Meta, etc. pp.

It only contacts those services if the users tap into it and I think people generally understand that those services have to be contacted to display content from them

It really depends on the market you're in. I'm coming from a European / GDPR centric POV where you'd have to have the user opt-in to this feature, even if you're only running it when the user clicks on it. But that's only for Europe (and California iirc).

haileyok commented 11 months ago

linking @seboslaw's post about the solution to that

https://bsky.app/profile/seabass.bsky.social/post/3khhzleejky2a

So in Europe you would have to show a disclaimer (on first click) that the app will contact an external 3rd party provider and will potentially transfer private data. Only if the user opts in you would then start the embed. Otherwise, you could offer to redirect to the external app.

Frankly might be a good idea anyway just to keep app review off of you.

pfrazee commented 11 months ago

If there's a regulatory requirement we should do it on first tap I figure. If you have to go into settings and enable it then nobody will.

haileyok commented 11 months ago

Agreed. It's non-obvious to go into settings to do it. Probably just display a sheet on first press letting the user know, then remember the setting from there.

Also to be fair, even without a regulation, I think people are attracted to the "less intrusive" aspect of federated stuff and giving warning is a good faith sort of thing.

mackuba commented 11 months ago

As a side-note. The app currently automatically opts-in users to this functionality. However, from a data privacy perspective this is kind of edgy since the app now per default (at least) leaks the user's IP address to 3rd parties like Google, Meta, etc. pp. So imho this feature should be opt-in and there should be a disclaimer informing the users what information is passed onto the 3rd parties when enabling the feature.

It only contacts those services if the users tap into it and I think people generally understand that those services have to be contacted to display content from them

Uh, so I have the exact opposite request, I'm currently seeing gray rectangles in place of thumbnails until I click on the box, and I want to see the thumbnail immediately ;)

pfrazee commented 11 months ago

yeah something seems to be inconsistent about the metadata fetch now, I gotta look into that

surfdude29 commented 11 months ago

If there's a regulatory requirement we should do it on first tap I figure. If you have to go into settings and enable it then nobody will.

As well as a short explanation shown on first tap, it might be a good idea to add a section to your Privacy Policy explaining the behaviour?

https://blueskyweb.xyz/support/privacy-policy

e.g. perhaps by updating and expanding this section:

IMG_8904

pfrazee commented 11 months ago

Opened an issue on the interstitial #2338

haileyok commented 11 months ago

A few things of note while implementing these.

  1. TikTok's embed API is a bit funky and - frankly - probably not a great idea for a few different reasons. Mainly, we would need to have an embed asset to provide to the webview. Probably not ideal at all, and frankly - especially on smaller screens - the experience would be pretty subpar anyway. Hard to display all the content thats inside that embed.
  2. I wonder what the utility of using imgur embeds would be. Unfortuantely, there isn't an easy way to tell if a particular link is going to be a video (mp4) or an image. I don't think there's any utility in having imgur image embeds (user should just upload the image to bsky) and I'm unsure how popular using Imgur for mp4 clip hosting is. We'd also need to use expo-av to display these which is honestly as bit clunky (even if we request a .gif or .webp for animated posts, those are still static)
  3. I didn't actually look into Instagram embeds, but I would assume that those are better left to the user to open in app. There's also the possibility that users will have embeds disabled for their posts, and without a good way for us to tell ahead of time, this is difficult to implement nicely.
WolfWings commented 11 months ago

As someone that's been active on imgur for over a decade, before giphy/tenor took over it was THE place (and still is for certain styles/eras) to post/store/share animated meme images, often tailored ones or highly customized ones, as well as supporting much higher resolutions than most other options, 20MB for static images, and up to 10k on a side, thus the source of the 'intergalactic' quality image macros.

It lacks any good search tools (lack of text being added to most posts, and the tag system being too limited and itself meme-ified as a result), but folks build extensive bookmarks/folder chains on-site and share them around to catalog images, it's basically the source of "Nice Meme, yoink!" type image-macros, etc. It also has a pretty extensive 'directly import video from other sites' toolkit built right in for making clips to turn into memes/image macros, etc.

It might be a little niche, but supporting directly embedding imgur content would be convenient for a lot of the reasons folks want to be able to post videos (yes, for the memes and other short clips), if you haven't been active on imgur across that whole decade-plus existence it can be an EXTREMELY opaque culture/site to explore due to how much of it is cyclically tied to specific dates as well for when certain content does/doesn't exist, etc.

Edit: One minor addendum also, Imgur's API does let you check if a given image is animated or not, and they've never supported anything except gifv or mp4 for video. Webp is just "better jpeg" for that site. The bandwidth/processing costs to convert everything to webm instead of just serving mp4 was enough they decided to just keep serving mp4 and gif as a fallback.

api.imgur.com/3/image/UID, with UID being the "BXsPWci" out of "https://i.imgur.com/BXsPWci.gif" for example.

Requires a header of "Authorization: Client-ID CID" with CID being a one-time client app registration. NOT a login against the site, no oauth required since it's just requesting public image data, so it'd just be "bluesky" globally or per-app or however you folks decided to organize things. https://apidocs.imgur.com/#authorization-and-oauth goes into that a bit more.

HeyItsLollie commented 11 months ago

Does Bluesky's YouTube embeds currently handle Playlist links? eg: https://www.youtube.com/playlist?list=PL8rSRNukjnn4GyPiGYY_wfnv03t8E131C

I'd also like to request:

haileyok commented 11 months ago

Does Bluesky's YouTube embeds currently handle Playlist links? eg: youtube.com/playlist?list=PL8rSRNukjnn4GyPiGYY_wfnv03t8E131C

I'd also like to request:

* [Bandcamp](https://bandcamp.com/) (Track & Album)

* [Sketchfab](https://sketchfab.com/)

* [Streamable](https://streamable.com/)

YT playlists doable.

Bandcamp embeds are tricky, would need to play with it and see what works. This is the embed URL: https://bandcamp.com/EmbeddedPlayer/album=3394138826/size=large/bgcol=ffffff/linkcol=0687f5/tracklist=false/artwork=small/track=3559172457/transparent=true/. The URL that you'd be sharing is: https://homesick89.bandcamp.com/track/1000db-stare. Notice that the embed URL is using an album ID while the share link is not. I'm not certain how we'd be able to get that or if it is possible. Have you seen anywhere else where these embeds are automatically taken care of?

For sketchfab, it's doable, but would want to see what the performance is like in a webview.

Streamable is actually super easy, and I think it is my preferred embed right now. Best video experience of the options. There's a blocker though with retrieving the thumbnails for them. I'll open a separate issue on that this week and hopefully it can be sorted out separately.

haileyok commented 11 months ago

api.imgur.com/3/image/UID, with UID being the "BXsPWci" out of "i.imgur.com/BXsPWci.gif" for example.

Requires a header of "Authorization: Client-ID CID" with CID being a one-time client app registration. NOT a login against the site, no oauth required since it's just requesting public image data, so it'd just be "bluesky" globally or per-app or however you folks decided to organize things. apidocs.imgur.com/#authorization-and-oauth goes into that a bit more.

Fortunately it isn't so much an issue of being able to get the info from the API when loading embeds (this wouldn't work anyway, their rate limit is far too aggressive for this purpose).

If we were always loading the content from the start, we wouldn't have any issue. expo-image has an animated value in the onLoad event that tells us if it's an animated image. So in that case, we would always use https://i.imgur.com/imageId.gif.

Instead, we first load a placeholder that is always not animated, and we have no information about whether it is or not. We'd probably need to make a request to the API endpoint you mentioned when posting the image and include info in the embed about whether it is animated or not. That's outside the scope of my current PR for various reasons (adding external APIs, modifying the embed metadata behavior, etc) so I can't really do it. But regardless I do agree they would be nice!

I suppose we could always just display the gif player even before we know if it is a gif or not, but that seems like a weird UX if you're thinking you're going to be playing a gif but then it doesn't animate...

HeyItsLollie commented 11 months ago

Notice that the embed URL is using an album ID while the share link is not. I'm not certain how we'd be able to get that or if it is possible,

Thankfully, it's all in the pages' <meta> tags, under bc-page-properties.

https://homesick89.bandcamp.com/album/exitminilp007-homesick-1000db-stare-mini-lp <meta name="bc-page-properties" content="{'item_type':'a','item_id':3394138826,'tralbum_page_version':0}">

https://homesick89.bandcamp.com/track/1000db-stare <meta name="bc-page-properties" content="{'item_type':'t','item_id':3559172457,'tralbum_page_version':0}">

Additionally, embeddable player URLs are provided upfront under og:video \ og:video:secure_url and twitter:player

og:video : https://bandcamp.com/EmbeddedPlayer/v=2/album=3394138826/size=large/tracklist=false/artwork=small/ twitter:player : https://bandcamp.com/EmbeddedPlayer/v=2/track=3559172457/size=large/linkcol=0084B4/notracklist=true/twittercard=true/

(note: tracklist=false and notracklist=true are completely interchangeable)

This embeddable URL is easy to manipulate and customize. For example, here's an album player with large artwork, tracklist enabled, a dark background and yellow links. The order of parameters doesn't appear to matter. https://bandcamp.com/EmbeddedPlayer/v=2/album=3541844592/size=large/artwork=large/bgcol=000000/linkcol=ffd200/tracklist=true/transparent=true/


Have you seen anywhere else where these embeds are automatically taken care of?

Tumblr, Twitter, and Cohost.

I believe Tumblr uses the og:video URL provided above, which matches Bandcamp's "Standard" player with "Small" artwork and "Show tracklist" disabled.

firefox - 2024-01-02 - 12-24-15

Twitter's embedded player seems to differ slightly from the twitter:player URL when visited directly, but Cohost uses Iframely to process its embeds, and the resulting embed virtually matches Twitter. (Minor style changes, eg: Twitter has a centered Play button, both have different link colors.)

Both appear to use Bandcamp's "Artwork-only" player.

firefox - 2024-01-02 - 13-42-40 firefox - 2024-01-02 - 13-54-19

haileyok commented 11 months ago

@HeyItsLollie yea i've opened an issue here about that: https://github.com/bluesky-social/atproto/issues/2009

right now there's no where to put that info nor is it provided in the metadata fetch response. if both of those are done, then we can really make this work! (and afaik both are rather trivial to do). but the lack of both of those blocks this for the moment.

we literally just had a thread about this on bsky. thanks for the great stuff!

WolfWings commented 11 months ago

I suppose we could always just display the gif player even before we know if it is a gif or not, but that seems like a weird UX if you're thinking you're going to be playing a gif but then it doesn't animate...

Actually a bit more digging...

...turns out they DO indicate the difference in the OpenGraph details.

https://www.opengraph.xyz/url/https%3A%2F%2Fimgur.com%2FTgdRHOd (still image)

https://www.opengraph.xyz/url/https%3A%2F%2Fimgur.com%2FBXsPWci (animated image)

Animated images always get a .gif for the twitter:image and og:image parameters, and likewise still images never do.

Avoids the API rate limiting and since opengraph is used for so much embedding details imgur exposing animated vs not via that rubrik should be usable I think? Maybe in the other issue you opened instead of this one, I'm good at tackling API nonsense but I admit I don't feel up to building the BSky app to test ideas directly in the codebase.

haileyok commented 11 months ago

Yea its 100% workable with that data, but just needs some changes both to the external record as well as the meta fetch util. Hoping that can happen!

YuriGal commented 10 months ago

If this is the right place to ask for additional players - would you consider adding YouTube Music https://music.youtube.com ?

Zero3K commented 10 months ago

Please add support for links such as https://packaged-media.redd.it/tajnlkezevdc1/pb/m2-res_472p.mp4?m=DASHPlaylist.mpd&v=1&e=1705950000&s=a066af59715c9eb450f69037e71c95541aac2565#t=0.

HeyItsLollie commented 10 months ago

I can already say there's no way Bluesky will implement embeds for direct links to video files. Why would you want to link to this file directly, instead of the Reddit post that this video originates from?

Zero3K commented 10 months ago

Because that is complicated. I take it that in order for video support to work, it has to be embedded from a different site which has to be allowed by the developers for the code to be added or hosted directly on one of bsky.app's servers.

steveharoz commented 9 months ago

I'd like to suggest a static fallback for sites like TikTok whose embed are a bit trickier.

Currently, including a link card from Tiktok just shows the root site's title, which isn't especially informative. Instead, Tiktok's oembed response (docs here) provides:

How about at least using that info for a static card that links to the video?

Laerios commented 9 months ago

In support of a decentralized ecosystem, I'd like to suggest PeerTube video embed support (its Embed API supports configuration via query parameters and also scripting).