Improve transcoding engine flexibility

[InfinityTotality]

Original Title: DSub max bandwidth setting not triggering transcoding

I've just deployed a Navidrome server via Docker image using the provided compose file, and I'm having an issue with automatic transcoding not occurring based on the client's max bandwidth setting. I'm using DSub (5.4.4) as the client, and I have it set to unlimited bandwidth on local network and 320kbps on mobile network. Here is the log entry from the Navidrome server when playing a song while connected via mobile network:

time="2020-06-14T02:32:58Z" level=info msg="Streaming file" artist="ミラクルミュージカル" bitRate=0 cached=false format=raw originalBitRate=899 originalFormat=flac title="Isle Unto Thyself" transcoding=false user=xion

I am certain that it is recognizing the network correctly as it is also set to only buffer 2 songs on mobile, which it is doing. I've also done a packet capture on the request for the song, and the URL does include a "maxBitRate=320" parameter:

GET /rest/stream.view?u=xion&p=[...]&v=1.2.0&c=DSub&id=92533410d46fb6018ce1b0678a3b53ae&maxBitRate=320 HTTP/1.1

With debug logging on, I notice that the server does recognize that requestBitrate=320, but then it goes on to just stream the raw file anyway. I have done some messing with the transcoders, but I have confirmed that it works if I manually specify a transcoder on the player in the Navidrome settings. Currently the only change from default is that the "-b:a %bk" in mp3 audio is replaced with "-q:a 0".

[InfinityTotality]

After looking through more logs with transcoders assigned and unassigned, I'm starting to suspect my expectation for how transcoding should be applied and the way the server is designed are not entirely aligned. My expectation would be that if the max bitrate is not set or higher than the bitrate of the file, no transcoding will be applied. The way it seems to work is that if a transcoder is assigned to a player, it will always run, and if none is assigned, it will never run. Is this the intended behavior? If so, can I formally request a feature be added to bypass transcoding if the bitrate does not need to be reduced?

[deluan]

What you are describing is called downsampling. I didn't implement it because I personally don't have a use case for it. The main focus for the current implementation is being able to force transcoding to happen for a specific client, as some clients do not have support for transcoding options and can't play all formats.

Please go ahead and create a feature request for it, describing your use case.

[InfinityTotality]

By that, you mean create a new, more appropriately titled issue?

[deluan]

Oh, nvm. I thought that's what you meant by "If so, can I formally request a feature be added to bypass transcoding if the bitrate does not need to be reduced?". But no worries, we can use this issue to track this.

I will give some thought to this and will come back here to discuss some solutions

[InfinityTotality]

Cool, thanks! Enjoying the server so far otherwise. Will likely be using this over Airsonic going forward

[slavmetal]

Are there any news on downsampling? 🙂

[deluan]

Hey @slavmetal, can you describe what is your need / use case for downsampling? I'm asking because I need to come up with a flexible way to cater for various disparate use cases.

[slavmetal]

Hey @deluan, for example it would be nice to reduce mobile data usage

[deluan]

Yes, but that is already possible with the current transcoding support. I'd like to understand what are the use cases not supported by the current transcoding implementation

[InfinityTotality]

If I didn't already describe this, my use case is that I use the same phone/player both at home and when I leave home. When at home and on wifi, I would like to just play the raw files (lossless, 1mbps+) for maximum quality and to avoid doing the extra work on the server to needlessly transcode them. When I'm not at home, cell data speeds mean that I can't reliably stream those raw files, so I need them to be trascoded down to 300kbps or so in order to actually stream them. There is no way currently for me to do both of those things without logging in to the web interface and manually changing the transcoding settings any time I change locations.

[deluan]

@InfinityTotality yeah I understand your needs, just wanted more example use cases to be sure I cover all (or most) of everyone's needs.

For your use case, I'm thinking of introducing a "force" flag to the player transcoding set, that would mimic the current behaviour. But when the flag is turned off, it would only transcode if the requested bitrate is smaller than the actual file bitrate. What do you think?

I've been thinking about rewriting the whole transcoding activation logic, that's why I'm taking so much time to implement this, but maybe the change I described above is enough for now.

[InfinityTotality]

That sounds like a perfect solution for me. I don't know how universal it is among players to have an option to request a particular bandwidth, so there maybe be some other triggers for the conditional transcoding that could be useful to other people, but I don't know enough about the ecosystem to know what those would be or how widespread they are.

[amil-m]

Just chiming in that this would suit my usecase too, I have struggled with getting ND_ENABLEDOWNSAMPLING, ND_MAXBITRATE and ND_DOWNSAMPLECOMMAND to work

[deluan]

Hey @amil-m these options are not available anymore, they were used before the final transcoding feature was in place.

To configure transcoding in the newer versions, you go to the Players view (in the UI), find the player you want to enable transcoding for, click on it and configure it:

[amil-m]

Thanks @deluan, I must admit, I overlooked this page!

[WeekendWarrior1]

Another use-case: I have most of my music in a lossless archive format (flac), and want to transcode it to something lossy (opus) when I am out of the house. But some of the albums I own were only ever released in lossy (mp3). When I am out of the house, I would not want to transcode these albums, as lossy -> lossy just means compounding/generational losses.

Airsonic lets you choose, for each transcode profile, a whitelist of file extensions you want the profile to transcode (eg. .flac, .wav, .ape, .alac)

PS. love Navidrome, great software

[kolmone]

It would also be useful to be able to set default transcoding settings for any new clients added. So for example any new client would transcode to mp3 at 320 kbps.

Perhaps it should also be possible to limit the transcoding options available to a user, for example if I created a guest account for my library I could limit them to only <=320 kbps streams.

[charleslaw]

But when the flag is turned off, it would only transcode if the bitrate of the song is smaller then the client's requested or the actual file bitrate, whichever is smaller. What do you think?

I'm not sure I follow this. Would this ever increase the bitrate? If so, I don't think increasing the bitrate does anything. I would think that once the song was encoded with a low bitrate, the quality cannot be improved.

[deluan]

@charleslaw, sorry the text was confusing. I fixed it:

But when the flag is turned off, it would only transcode if the requested bitrate is smaller than the actual file bitrate.

[bhartshorn]

@deluan Navidrome is great, thanks for your effort!

When at home and on wifi, I would like to just play the raw files (lossless, 1mbps+) for maximum quality and to avoid doing the extra work on the server to needlessly transcode them. When I'm not at home, cell data speeds mean that I can't reliably stream those raw files, so I need them to be trascoded down to 300kbps or so in order to actually stream them.

I have the same use-case. ^^^

For your use case, I'm thinking of introducing a "force" flag to the player transcoding set, that would mimic the current behaviour. But when the flag is turned off, it would only transcode if the requested bitrate is smaller than the actual file bitrate. What do you think?

This would work just fine for me.

It might be nice to be able to insert the requested bitrate value (with a fallback to the value set in Navidrome config) into the transcode command.

[NyaomiDEV]

Citing my message on the Discord server:

In my case, I hold everything in lossless format (flac/wav mixed ranging from CD quality to 32bit / 192khz). I want to be able to stream flac 44.1khz 16bit music to everyone and then offer a download button for the full-res track(s). Then, I want to have a guest user which cannot download and which is forced to listen compressed OGG music (in my case I need to value network speeds over computational overhead, so it makes sense that the guest is not really treated on par with regular users)

The transcoding engine should be flexible enough to force a specific transcoding setting to an user

[mill1000]

Throwing my opinion into the fray as a new user.

• I agree with the above mentioned mobile vs home use cases.
• I want the ability to set a default transcoder for new players. Added bonus if it could be a per user settings. This way less savvy users could have a bandwidth friendly default, and advanced users could switch to lossless streams as they desired.

Additionally, is there any documentation on the function of the Max Bit Rate setting? I couldn't quite grasp it's purpose, and how it interacts with the Default Bit Rate setting.

[deluan]

@mill1000:

Additionally, is there any documentation on the function of the Max Bit Rate setting?

No, there's no documentation and I don't even remember what exactly it does... 😛 I think this card is over due and I plan to work on it pretty soon.

[mill1000]

Ha ok, well I won't stress about it then.

Some other thoughts I had on transcoding:

Arbitrary quality levels The current implementation pretty much assumes constant bitrate (CBR) encoding. We can enter arbitrary commands so it's still possible to use variable bitrate (VBR), but we lose the ability to adjust the quality without setting ND_ENABLETRANSCODINGCONFIG and editing the command.

This could be worked around by writing a wrapper script that converts the CBR values into an approximate VBR quality level but I think we should handle it in Navidrome instead.

One approach, instead of fixed bitrate values we should have arbitrary quality settings.

Each setting would have:

• A name: the displayed "friendly" name.
• A value: the value that is substituted into the command line.
• An effective bitrate: this effective bitrate is used to map CBR values to these new arbitrary quality settings. e.g. if a client asks for 192 kbps, Navidrome can use the effective bitrate to find an equivalent quality setting.

Example settings for VBR MP3 using -q:a 1, -q:a 4, and -q:a 7. Estimated bitrate from here.	Name	Value	Bitrate
High	1	220 kbps
High	4	162 kbps
High	7	100 kbps

Duplicate format types The current implementation requires the target format to be unique. This prevents us from defining different transcoders that output the same format. e.g. I can't configure a VBR MP3 transcoder and CBR MP3 transcoder, or 2 different VBR MP3 transcoders (e.g. a "high" quality one, and a "low" quality one to work around the above mentioned limitations).

[tokyovigilante]

Just adding in as requested in #1312 a request for custom format support, in my specific case I have a bunch of MOD/AMF/S3M tracker-based music which could be imported into the library and either passed as is to players supporting the format, or converted to opus for streaming/browswer playback by piping PCM output from the decoder through opusenc.

Additional info in the other ticket, but I had envisaged using the extension or mime type to select the decoder as follows:

".MOD,.S3M,.AMF": "openmpt123 -q --stdout --no-float <input> | opusenc --raw --bitrate 256 - output.opus"

This could obviously be extended to other formats as required, and have differing options based on local network (direct play) vs mobile (transcoding).

[snowphone]

I'm not sure I'm too late to introduce my case. My music collection is migrated from iTunes, so there's some alac (apple lossless m4a) files. The problem is, almost none of subsonic-api based player can handle this codec. So, these files should be transcoded to flac, opus, or something. One may suggest to convert alac files to flac ones, but it's not that easy since keeping metadata (playcount, star, rating, playlist...) is the problem. The rest is same as ones who are struggling with big flac files (#794).

So, file extension + bitrate based downsampling (transcoding) is good enough, but codec (to differentiate alac from aac) + bitrate would be perfect, it should be long-term goal though.

[AlmightyFrog]

I am not sure if i just missed it or didn't get it so far: My usecase for transcoding is maybe similar to tokyovigilante and snowphone to support playing some "special" formats at all. Here it is some WMA files, which i want to have transcoded, but nothing else.

Would it be possible to make the transcoding conditional depending on file type (best would be a blacklist what not to transcode for me)?

[ampcat3]

Another transcoding use case would be to have ffmpeg use the replaygain tags stored in audio files in order to "normalize" the output volume. This is done with -af volume=replaygain=track or -af volume=replaygain=album. I don't need to have the bitrate or audio format changed so the option to force the transcoding to be used on all files would be beneficial.

[J4gQBqqR]

My data gets drained out this month by DSUB because all my files are in flac and no transcoding happens. 😂

[metalheim]

Just adding my thoughts:

There should be client-side compatability checking #2051

In addition to that, it makes sense to introduce "transcoding/compatability profiles" that can be assigned to a "Player". A Transcoding profile would contain a list of known-compatible and known-incompatible formats. This is f.e. how https://github.com/jellyfin/jellyfin handles file compatability with a vast base of different clients.

If a known-incompatible file is attempted to be played, navidrome would force transcoding to a known-compatible format.

Navidrome should ship with a handful of default profiles, like "iphone", "android", "legacy browser" and "modern browser". "modern Browser" would probably have a compatability list of:

Known Compatible: "audio/mpeg" "audio/ogg" "audio/mp4" "audio/flac" "audio/x-wav" etc.

Known Incompatible: "audio/x-musepack" "audio/x-monkeys-audio" "audio/x-shn" "audio/x-ms-wma" "audio/dsd" "audio/mp4", lossless: true etc.

[hischampion]

I would also love to see this done. My opinion, in order

1. Client codec compatibility list vs file codec. This would have a default quality setting, bitrate, or full command line string. Config value
2. Compare file max bitrate vs client preference

Transcoding would happen if required by the above rules. Bitrate chosen overruled by #2. Transcoding done once.

I think adding defaults for new users, a pipeline to add replay gain or even #2 should be separate work. Iteration over perfection.

The part I’m not sure how to do is identify the clients reliably but maybe there’s a good way users can do it themselves.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Navidrome team are limited, and so we are asking for your help. If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open. If this is a feature request, and you feel that it is still relevant and valuable, please tell us why. This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

[Extarys]

Still relevant.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Navidrome team are limited, and so we are asking for your help. If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open. If this is a feature request, and you feel that it is still relevant and valuable, please tell us why. This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

[ne0-wu]

I think having an option to enable transcoding by default would be quite beneficial. This feature may improving streaming consistency and reduce data usage for mobile data users and budget-friendly servers with limited bandwidth capacity.

[aqxa1]

Another issue with the current implementation is that "Max. bit rate" in the UI when forcing a player transcoding preference, doesn't seem to be a maximum but rather the default.

E.g. if I set DSub preference to 192kb @ opus in the server UI, then it will correctly encode to 192kb @ opus if DSub max bitrate is set to Unlimited or to 192kb. But if I set DSub to 320kb, then it instead uses 320kb, even though the Navidrome maximum is set to 192kb.

I'm not sure if this is just a bug, but if not, something like "default bitrate" or just "bitrate" in the UI might be clearer.