damus-io / damus

iOS nostr client
GNU General Public License v3.0
1.95k stars 290 forks source link

Media Optimization #2214

Open jb55 opened 1 week ago

jb55 commented 1 week ago

This is a tracking issue for media optimization-related tasks

Transcoding/video uploads:

@fishcakeday has been doing some work on the nostr.build side, so we shouldn't try to overlap work where we don't need to. Even is that case, we should still be looking at optimizations for non-nostr-build media sources via proxies.

fishcakeday commented 1 week ago

May I suggest to start with doing the initial transcoding of the video on the client: https://developer.apple.com/documentation/avfoundation/media_reading_and_writing/exporting_video_to_alternative_formats

Then pick the correct quality: https://developer.apple.com/documentation/avfoundation/avassetexportpresetmediumquality - is a good one for this purpose.

Resolution: https://developer.apple.com/documentation/avfoundation/avassetexportpreset640x480 - 480p is universally great and perfect for a phone sized video in 99% of cases. I do not advise going over 720p for this purpose.

Ensure to set this: https://developer.apple.com/documentation/avfoundation/avassetexportsession/1390593-shouldoptimizefornetworkuse - this moves the important bits to the beginning of the file and allows quick start of the playback

You could also enforce: https://developer.apple.com/documentation/avfoundation/avassetexportsession/1622333-filelengthlimit - since NB limits free to 10MB, but no limit for users with account. (Optional)

There are many options that can be used to make videos great and quick without any proxies if they are uploaded from Damus, AFAIK Amethyst already doing it, but the quality is unwatchable and not well chosen.

jb55 commented 1 week ago

@fishcakeday thanks I broke these into tickets for tracking