search

jellyfin / jellyfin-androidtv

Android TV Client for Jellyfin
https://jellyfin.org
GNU General Public License v2.0
2.66k stars 455 forks source link

The Playback Rewrite #1057

Open nielsvanvelzen opened 3 years ago

nielsvanvelzen commented 3 years ago

:rocket: What is The Playback Rewrite?

The goal of the rewrite is to create new playback code from scratch for video playback, audio playback and live tv. The new code should be easier to maintain and be prepared for a migration to Jellyfin 11. New features like SyncPlay can be added without much trouble. All code is Kotlin-first and uses the new SDK. It should be used by all our official Android apps for consistent behavior between (Android-) devices when playing files.

The rewrite is a very long project because I'm just one person volunteering to work on Jellyfin in my spare time. There is no hard deadline or specific version goals. This issue contains a task list of the bigger features/changes.

Status updates will be posted occasionally in the comments. Please read the bottom paragraph of this issue before commenting yourself.

:trophy: Tasks

This list will change over time, some features might not be available on the initial release of the new playback code. When possible issues/pull requests will be linked for tracking purposes. There is no particular order for this list, new features might be added before restoring older features.

The amount of checked boxes does not indicate progress or completeness of the rewrite. Some changes required to make this happen have been running in production since version 0.15.0.

:building_construction: Preparations

Various tasks that need to be completed to work on the rewrite.

:fireworks: Proof of Concept

:arrows_counterclockwise: Restoring features

:hammer_and_wrench: Integration (ATV)

:hammer_and_wrench: Integration (mobile)

:boom: Adding new features

Server blockers

Some features cannot be made or won't work great without changes to the server. This is a list of them.

:exclamation: This issue

Please do not reply to this issue for any questions about the playback rewrite. The comment sections is meant for our contributors to give updates or discuss the implementation. For anything else, please create a new issue or discussion. We may delete your comment if we consider it off topic.

mueslimak3r commented 2 years ago

I was looking intro the code for trailer playback last night and I see that the object for external trailers just uses links videos (eg: a link to a youtube video). I'm not sure how feasible it would be, but it seems like a good design for a re-written player might be for it to just play video links. Then other classes would be queried for stuff specific to jellyfin like subtitle object classes, mediastream info, etc.

With the current player it's entirely designed around using jellyfin specific classes like BaseItemDto and its MediaStreams and SubtitleInfos, etc. Because of this it's not possible to make exoplayer play an arbitrary youtube link, even though exoplayer is capable of doing so.

nielsvanvelzen commented 2 years ago

it seems like a good design for a re-written player might be for it to just play video links.

The current implementation already does this, it never uses the Jellyfin API directly to make it easier to update if the ever API changes. This doesn't mean that YouTube video is supported though, the implementation will only allow video from the Jellyfin server. If we want to support YT a server plugin should deal with that, not the client.

Then other classes would be queried for stuff specific to jellyfin like subtitle object classes, mediastream info, etc.

My idea is to create some kind of "lookup" service where the playback code queries for data. This service would be implemented in the app code. I haven't worked on this part yet.

it's not possible to make exoplayer play an arbitrary youtube link, even though exoplayer is capable of doing so.

I don't think this is possible without using the proprietary YouTube API.

Andy2244 commented 2 years ago

Just a quick note regarding your thoughts on "Rethink external player support".

The most interesting players that have better or more features than Exoplayer nearly all have a basic API, so its mainly a matter of implementing the available api. So my suggestion would be to define a clear interface that can be implemented for a particular player. So as example there is often no need to use a external player for audio, while you do want to use it for video. So having a interface to probe what is implemented and whats not may help.

So from my perspective Exoplayer/libVlc should be handled like a external player, but with fully implemented interfaces as reference.

Just a suggestion.

nielsvanvelzen commented 1 year ago

Quick update for everyone watching the progress on this rewrite. I'm still working on it! Even though the issue was made quite a while ago I've made a lot of progress but also started over a few times. I want to get the basics right so we don't end up with unmaintainable code.

My focus is currently on getting audio-only media to work perfect before proceeding with video support. This will probably mean that eventually the app releases an update with the new playback code for music/audiobooks and the old playback code for series/movies.

The following list of features are working locally already, there might even be more I forgot to mention:

There's still a lot of bugs and some design decisions to make in the overall architecture. Hopefully this it won't be too long before the first pull requests with actual code go up. Thanks everyone for waiting!

gitdeath commented 1 year ago

Would the Media Session Support mentioned be the solution for #441? If so, is this something that could be included in one of the 15 releases, or is it gated behind other things in this project?

nielsvanvelzen commented 1 year ago

Yes media session support will fix #441 and no they're not easy to integrate in the existing playback code. If everything goes right the rewritten playback code will be available in 0.16 for music playback (no promises).

nielsvanvelzen commented 1 year ago

I've updated the initial post; removed some unnecessary info. Rewritten the intro and added a list of changes required in the server.

The past few weeks saw quite some progress on the rewrite (see the linked PR's above this comment). The current plan is to use the new playback code for audio playback (music/audiobooks) first before working on video again. This is mainly because it's easier to get the architecture started and have some working code.

Music playback is already working for me locally and I'm slowly cleaning all code and making pull requests to get it into the main repository.

Thanks for being patient with my 2 year rewrite project 😄

nathangur commented 1 year ago

Yeesh. 2 years ago and 12 have been finished

nielsvanvelzen commented 1 year ago

Hello everyone, I'm back with another status update!

The last few weeks saw a bunch of changes (see pull requests above this comment) and I can now say: we have reached feature parity with the old music player 🎉. Today I merged a change that uses the new playback code as default for music (and other stuff using the audio player).

Work will continue to add additional features, improve performance and fixing bugs. I will also accept pull requests to the new playback code now. Please discuss first with me if you're planning to make changes.

When 0.16 is released I will start work on video playback so we can hopefully fully replace the existing player code in 0.17/0.18 (as always; no promises). I plan to start the beta cycle for 0.16 in the coming months.

PS: a quick reminder to read the issue description before replying, your comment is likely to be removed/hidden. Also read our Code of Conduct before complaining to unpaid volunteers.

nielsvanvelzen commented 1 year ago

The first beta for 0.16 is here!.

As said in previous posts, this release uses the rewritten playback code for all music playback by default. Like all beta releases it will be available in the beta program on Google Play only when it's been trough review.

If you encounter any issues you can (temporarily) switch back to the old backend from the app preferences, please report your findings on GitHub (in a new issue) so I can fix the bugs before release.

I'm already working on the initial changes for video support locally which I'm hoping to make a part of 0.17 (no promises!) which I'm hoping to release before the end of this year (also no promises!).

If you appreciate my work, you can support me by donating via GitHub sponsors. I use these sponsorships to buy equipment for my development machine and hardware to test specific issues.

thor2002ro commented 11 months ago

even if av1 still doesnt work in the client on amazon fire tv 4k max that has av1 this check "Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q" in jellyfin/androidtv/util/profile/MediaCodecCapabilitiesTest.kt will fail since the stick reports android 9

adb shell getprop ro.build.version.sdk 28

adb shell getprop ro.build.version.release 9

Yeetov commented 4 months ago

Is this the same code for the fire stick (firetv) or just android TV?

nielsvanvelzen commented 2 months ago

Let me give all of you another (quick) status update about the famous playback rewrite. I've been very busy the past few weeks (might even be months, didn't check) on a few key points:

  1. 10.9 compatibility. This work happens in multiple places, from manually validating each API change in the release to updating the SDK and updating the app. This work is now done and the current 0.16.z releases of the app are fully compatible.
  2. Refactoring the app to fully use the SDK. I recreated the layer that talks with the API a few years back, this is our "Kotlin SDK". However, the app still used our old "Java Apiclient" in most places. With all new additions in 10.7, 10.8 and 10.9 I could no longer not use the SDK as all newly added features could not be used. This migration took up most of my time and I can now say that almost all code is using the SDK! The first (real) 10.9 feature I added is support for normalization gain in music playback (#3660)!
  3. Rethinking some core components of the playback code. Most of this is now implemented and this should make it easier to add video playback related functionality.
  4. Add video support. The initial changes are already merged, and development builds have an option to enable it. It's highly experimental but it does work, you can play video now.

While all of this is great news, I'm also working towards the next release of the app, 0.17.0. The current release started its beta cycle almost a year ago so it's time for a new one. Considering the new video playback is still very early it will not be ready for 0.17. The experimental option for it will also most likely be hidden for the release (still debating this part).

This was the quick status update for now, I didn't bother too much with the formatting or even re-reading it so forgive me for any mistakes. Hoping to give the next status update a lot quicker!

Happy watching!

If you appreciate my work, you can show your support with a donation through GitHub sponsors.