Feature request: Possible to add streaming of video files but only audio?

[apjyotirmay]

Hi, I'm wondering if it's feasible and possible to add an option to select video libraries and stream just their audio part?

[ryanwwest]

I'd like this too. I have some music videos and would rather not create audio-only duplicate files of them, as when I'm listening to a Playlist I have a mix of audio songs and a few music videos. You'd have to favorite both the music video and audio song of the same thing, for example, for your library to be consistent. Currently though, streaming audio only of video files I think has worked a few times in the past when online, but doesn't work when offline. Additionally, it doesn't allow me to play a Playlist with a mix of audio and video files at all.

[Chaphasilor]

Online support for playing music videos is mostly there, but the library in Jellyfin needs to be of type "Music Videos" for it to work. Offline mode / downloading support depends on #215 / #496 to only download the actual audio and not the whole file.
I'm not sure about the issues with playlist and don't have the capacity to look into it myself right now 😅

[Chaphasilor]

could you try if this is working with the latest beta version?

[ryanwwest]

I updated to latest beta version on Google Play today and actually the app totally crashes on boot now. It says:

"Something went wrong during app startup. The error was: Unknown baseItemDto type Video"

So to use it I would need to downgrade, can't speak to the issue above.

[Chaphasilor]

Oh okay, that is not good. Sorry about that!

How exactly are your libraries set up? It seems like you have actual Videos, not MusicVideos?
You can also try resetting the app by by clearing the app's storage in settings. What you're seeing is probably a migration bug, and it might work after clearing things. And it could allow you to grab some logs for me to look at.

[ryanwwest]

Resetting storage worked, I don't have the error anymore, thanks. I have to redownload my library, but at least when streaming from server direct, a music video added to a playlist I have with audio is working (which I think was also working before). I cannot search for that music video in any Finamp search areas, only access it via the playlist. I am downloading the whole library and once it finishes, can report back if whatever was downloaded for that music video (audio only?) still works or if it breaks that playlist when offline as it previously did.

Perhaps I cannot see the music video outside the playlist it is a part of because in Settings -> Select Music Libraries, I only have my 'Music' library selected (there is also Movies, Music Videos which I want to select but it is greyed out, and Shows). So playlist is a way to bypass the fact that the Music Videos library is missing.

[ryanwwest]

I tested adding a regular video (not from Music Videos) to the playlist. It actually played audio if I shuffled the playlist as the 'video' was eventually picked, and chose probably the default audio track to play. Interestingly though, the playlist view didn't even show the video nor a music video added after it. It gave the correct count of songs but hid them unless reached from playing the whole playlist and moving to the next item. When removing the video, the music video showed up again in the list.

[Komodo5197]

The playlist view breaking is probably due to the download system not knowing what to do with the Video, similar to how it also broke the migration. We can probably just add a case to treat them identically to MusicVideo. Also, as you noted the MusicVideo is not in the Music library, so downloading that will not grab it. You need to specifically download the playlist, or individually download the music video, to see if it works offline. You may also want to try downloading it transcoded, as I'm not sure anything would have changed from stable to beta regarding playing back the original file offline, if that wasn't working before.

[ryanwwest]

Makes sense. Are there plans to be able to select libraries of type MusicVideos as well to be played/downloaded? I don't think it matters for the other ones, and I only think audio would matter but it would be nice to search music videos in songs and genres (and Artists, when offline support for that comes out).

[Chaphasilor]

I think we could support music video libraries if playback is working, but we'll have to see how to handle it properly. Music videos are a different Jellyfin type with different properties, and it could be that some information can't be shown or errors are thrown if we try to treat them like the Audio type.

If you're on Android, I could compile you a custom version that lets you select music video libraries so you can give it a try. But I'm not sure how Finamp will behave in that case...

[ryanwwest]

Could the custom version be installed alongside the regular one, or will they clash? I'm happy to help beta test

[Chaphasilor]

Okay, here's the custom version: https://gofile.io/d/raLzHX
You can install it alongside the regular/beta version because it uses a different application ID (ending in .musicvideos). It's also signed with my certificate and not James', so you couldn't install it on top of your current version anyway, Android won't let you. This version also won't receive any updates for the same reason. It also has the same name, but I think you'll manage.

It seems like music video support in Jellyfin isn't great. The web client simply shows a list of all videos, and anything related to music (artists, albums, etc.) is not available. I've made it so the videos are requested only on the "Songs" tab, so only the songs and playlists tabs will work.
Let me know how it goes!

[ryanwwest]

Well, no crashes :) Within the constraints you set it seems to be working well when exclusively using playlists (so far with a playlist of a mix of songs and music videos on repeat, they all play correctly when they didn't used to a year ago, though when relooping the playlist it autopauses so I have to unpause it). But a few problems:

• I cannot play music videos directly from the Songs tab (nothing happens tapping them). I can play them by adding them to 'Next Up'. There are also fake/null items with the name of each folder within MusicVideos collection, and another fake/null item called 'musicvideos' maybe for the entire thing - clicking these just plays a random non music video. When Offline mode is enabled, the Songs tab works correctly for downloaded songs.
• When trying to download the entire Music Videos collection (by opening the left sidebar in 'online mode' and clicking the download icon to the right of 'Music Videos'), it doesn't seem to attempt to actually download any of them. Playlist downloads work well, as they previously did.

Genres tab actually shows two genres as some of my music videos have Genres assigned. But they show all songs from every collection, not just Music Videos. I guess downloading that is somewhat of a workaround for downloading the entire collection not working.

It would be nice to search any song in any collection, but the only way I know to cross-collection is in Playlist tab for items that were added across collections. In Genres, search only seems to search genre names, not names of songs within. I know the standard Jellyfin Android app allows a 'global' search but I'm sure it'd be different for just the collections you specify, and different again for offline mode.

In the main Jellyfin app, I see you can 'Edit Metadata' of a music video and see its Artists, Album and Genres. I'm not familiar with the code but wonder if these could eventually be access in Finamp. MusicVideos is also similar to TV Shows collection in that you can additionally organize musicvideos into folders, which I do. Perhaps these could be added as a new tab for just collections of the Music Videos type, with options to shuffle or download all items in a folder similar to album/playlist. If it is difficult to support Artist and Album for Music Videos, Folder only might be a first step at a minimum viable product to release, then worry abou the others later (as Jellyfin itself doesn't provide a lot of usability of Music Videos in Artist/Album either).

[Chaphasilor]

Okay, thanks for your detailed thoughts.

Playing music videos from the songs tab doesn't work because in the beta that would start an "Instant Mix" for the selected song (instead of the old behavior that would only play a single song and then stop). It seems like that's another thing that isn't supported for music videos.

Downloading the collection not working is expected, as @Komodo5197 pointed out. When you download a playlist, does it actually download the videos? And if you check the size on the downloads screen, is the video stream downloaded too (large download size)? What happens if you enable transcoded downloads (in transcoding settings) and then re-download the playlist?

How do you assign genres to music videos? I didn't know that was possible. The genres showing all albums and not just the ones inside that genre is a separate, known bug that also happens with artists. I believe it also happened in the stable version, but haven't looked into it yet.
If the genre albums were limited to just the music videos collection, it would probably be empty, because there is no album support for genres. Or is there also a (proper) way to do that, like for genres?

Global search is planned, as is some sort of filtering. But that is also a fully separate issue.
Same for editing metadata.

Adding folder support (in general, not just for music videos) is conceivable, but something I'd be willing to look into myself at this point.
Finamp is first and foremost a music client, supporting playback of music videos would be nice, but I'm not trying to add proper support for them at the moment.
I'd just like to make it possible for people like you to play them without major issues, if desired.

[ryanwwest]

When you download a playlist, does it actually download the videos? And if you check the size on the downloads screen, is the video stream downloaded too (large download size)? What happens if you enable transcoded downloads (in transcoding settings) and then re-download the playlist?

I'm not sure if the video is also downloaded (or where to find that on the system, 'Download Locations' setting currently shows nothing so there must be a default path that I don't know), but it probably is, as the filesize is 53MB instead of 54MB on the server for one video (which could just be attributed to filesystem or streaming differences). Enabling transcoding gave same download size for it and other musicvideos.

How do you assign genres to music videos?

In main Jellyfin web or Android app, you can use 'Edit Metadata' to delete and add Genres closer to the end of the options. But I don't know if the Jellyfin devs put a lot of thought into Genres/Artists/Albums for music videos as they exist but aren't utilized very much in a Music Videos collection. The current behavior in main Jellyfin is that when showing an Album page, it shows the regular songs which you can shuffle or play all, and below shows Music Videos that you can also play (with video) but are not added to queue when pressing shuffle/play all. It would be nice to have an album with combined music videos and videos show and shuffle through both, there and in Finamp. For completeness, current behavior in main Jellyfin for playlists is that both songs and music videos are included when shuffling/playing all, and both work (though the first time you play a music video, it switches to the video player UI and thereafter all songs also use the video player UI but a black screen).

Adding folder support (in general, not just for music videos) is conceivable

I was thinking Music doesn't use folders in main Jellyfin, except possibly for somehow inferring the album/artist. Can you use them in other ways?

Finamp is first and foremost a music client, supporting playback of music videos would be nice, but I'm not trying to add proper support for them at the moment.

I appreciate any support, it doesn't need to be major. I'm not too concerned about full video being downloaded for now - music videos are usually pretty short anyway. The only nice-to-haves for a base level of usefulness to me would be easy ways to mass download and play them online and offline through the Songs and Playlists tabs.

I tried something new, by downloading a Genre including both music videos and songs from the Music Videos collection. Unlike playlists tab, which seems identical for both Music Videos and Music collection, the Songs tab becomes different even after a restart after all downloads complete. For Music collection, only the downloaded songs belonging to a playlist appear. Under Music Videos, ALL music AND music videos under the Genre appear. So it seems the Genre download thinks it only applies to Music Videos yet displays non music videos.

Genre tab could just be disabled in Music Videos to hide these bugs for now if this would take too much time otherwise.