HDR support for the Steam Deck

alon-z commented 7 months ago

Is your feature request related to a problem? Please describe. Users cannot enable the HDR option on the Steam Deck (Linux client) when connected to an external display that supports HDR.

Describe the solution you'd like Being able to enable HDR setting in moonlight and see HDR content when streaming

Describe alternatives you've considered Possible to use other clients for HDR (couldn't find one that supports HDR 4K 60hz)

Additional context I have seen something about ffmpeg 6.1 that was released that enables HDR on vulkan? (I think) I would love to help coding (golang knowledge) but I have 0 idea on where to start here.

Thank you :)

Kabraxist commented 7 months ago

Seconding this. New Steam Deck OLEDs have builtin HDR support for its new display, but in the meantime Valve enabled HDR for first generation devices as well when connected to an external display that supports HDR. It seems to work well except Moonlight on Linux doesn't allow toggling the "HDR mode".

cgutman commented 7 months ago

On Linux, HDR is only currently supported when streaming outside of a desktop environment (where Moonlight can take full control over the display hardware to configure HDR). In theory, it should be possible to support HDR using our existing EGL+GLES renderer, but I think Gamescope only supports HDR via Vulkan.

In any case, I am working on a Vulkan renderer that will support HDR on Linux from within Gamescope.

kaitolucifer commented 7 months ago

https://github.com/streetpea/chiaki4deck/issues/93 I think this could help.

Jordan-Ch commented 7 months ago

I use moonlight regularly to stream from my PC to my Nvidia Shield, often with HDR.

Recently grabbed an OLED Steam Deck hoping to stream with HDR to the Steam Deck. Is this currently not possible?

HDR option on the Linux build seems to be greyed out.

eliw00d commented 7 months ago

I use moonlight regularly to stream from my PC to my Nvidia Shield, often with HDR.

Recently grabbed an OLED Steam Deck hoping to stream with HDR to the Steam Deck. Is this currently not possible?

HDR option on the Linux build seems to be greyed out.

Both the LCD and OLED Steam Decks now support HDR to external displays. Only the OLED supports it on its own display. However, the support for external displays is limited to games run on the Steam Decks themselves. Streaming apps like Moonlight need to add support for HDR separately. I hope that helps.

HA55EHH commented 7 months ago

It's the only thing holding back the Steam Deck for me at this point... hope you can figure it out!

david-pili commented 7 months ago

Not sure if you accept bounties or sponsorships for features but I'd happily pay to have this prioritized.

arshiatn commented 7 months ago

On Linux, HDR is only currently supported when streaming outside of a desktop environment (where Moonlight can take full control over the display hardware to configure HDR). In theory, it should be possible to support HDR using our existing EGL+GLES renderer, but I think Gamescope only supports HDR via Vulkan.

In any case, I am working on a Vulkan renderer that will support HDR on Linux from within Gamescope.

Guys from chiaki4deck got the HDR working and are debugging. I have tested it and it works great. I don't know how they made HDR work but you can check it out:

https://github.com/streetpea/chiaki4deck/pull/104

ritmototal commented 7 months ago

Also dying for HDR remoteplay on the steam deck so I can drop the nvidia shield and its horrible controls with moonlight due to its limited control configuration support. If the steam deck could do HDR remote play then it can be the ultimate TV Docked and Portable experience paired with a gaming PC.

zachrd commented 6 months ago

Happy to test this right now! Let me know what if any logs i can provide.

zachrd commented 6 months ago

Ok, initial test of the latest automatic build.

Cannot enable HDR within moonlight settings on the OLED Deck.Happy to try anything else needed.

cgutman commented 6 months ago

Yep, that's expected if you're using the AppImage build from CI. It doesn't have the required libraries yet for Vulkan. I'm working on that now.

cgutman commented 6 months ago

You can try this AppImage - https://ci.appveyor.com/project/cgutman/moonlight-qt/builds/48754905/job/7r54vi0b48f1b6n1/artifacts

You'll need to make sure you run it from within the regular Steam Deck UI rather than the desktop mode since it requires Gamescope for HDR.

There are still known issues with the Vulkan renderer but I think it should work for testing.

TheBacon00 commented 6 months ago

You can try this AppImage - https://ci.appveyor.com/project/cgutman/moonlight-qt/builds/48754905/job/7r54vi0b48f1b6n1/artifacts

You'll need to make sure you run it from within the regular Steam Deck UI rather than the desktop mode since it requires Gamescope for HDR.

There are still known issues with the Vulkan renderer but I think it should work for testing.

I actually just tried this build on my Steam Deck (from the "Game Mode" UI by adding the AppImage as a non-Steam game), HDR option is still greyed out. Do I need to set an env var or something to enable the new Vulkan renderer?

zachrd commented 6 months ago

You can try this AppImage - https://ci.appveyor.com/project/cgutman/moonlight-qt/builds/48754905/job/7r54vi0b48f1b6n1/artifacts You'll need to make sure you run it from within the regular Steam Deck UI rather than the desktop mode since it requires Gamescope for HDR. There are still known issues with the Vulkan renderer but I think it should work for testing.

I actually just tried this build on my Steam Deck (from the "Game Mode" UI by adding the AppImage as a non-Steam game), HDR option is still greyed out. Do I need to set an env var or something to enable the new Vulkan renderer?

Going to test the JSON here: FrogTheFrog

@FrogTheFrog posted this on the Discord the other day.

FrogTheFrog commented 6 months ago

You can try this AppImage - https://ci.appveyor.com/project/cgutman/moonlight-qt/builds/48754905/job/7r54vi0b48f1b6n1/artifacts

You'll need to make sure you run it from within the regular Steam Deck UI rather than the desktop mode since it requires Gamescope for HDR.

There are still known issues with the Vulkan renderer but I think it should work for testing.

I have tested AppImage (and my flatpak build), both with no luck.

I will generate some logs from AppImage in gamemode and add them here.

arshiatn commented 6 months ago

You can try this AppImage - https://ci.appveyor.com/project/cgutman/moonlight-qt/builds/48754905/job/7r54vi0b48f1b6n1/artifacts You'll need to make sure you run it from within the regular Steam Deck UI rather than the desktop mode since it requires Gamescope for HDR. There are still known issues with the Vulkan renderer but I think it should work for testing.

I actually just tried this build on my Steam Deck (from the "Game Mode" UI by adding the AppImage as a non-Steam game), HDR option is still greyed out. Do I need to set an env var or something to enable the new Vulkan renderer?

Same. I just added Applmage to Steam after making it executable and HDR (Experimental) is grayed out. I thought that I may have to connect to my PC first then change the options and it is still grayed out. I tried out every combination of Display Mode X Video Decoder X Video Codec and HDR is still grayed out. Restarting the steam deck didn't do anything either.

Using Windows 11 with a LG C2 OLED and (HDR on before connecting to my PC).

I have no idea how to generate logs for you but it seems like that @FrogTheFrog is going to generate his logs.

FrogTheFrog commented 6 months ago

So here are some logs (PLVK_LOG_LEVEL=5, for debug logs) for SteamDeck running from gamemode: with_prefer_vulkan.log with_prefer_vulkan_and_force_vaapi.log without_prefer_vulkan.log

I noticed there is SDL Error (0): pl_map_avframe_ex() failed, but have no idea what this means :/

matte-schwartz commented 6 months ago

This latest AppImage is a winner! Very impressive work, devs. System details: Nobara Linux, Kernel 6.6.7-200, Steam Deck OLED. Added the AppImage as a non-Steam game, switched to game mode and HDR was available as an option within Moonlight’s settings.

johnyz333 commented 6 months ago

This latest AppImage is a winner! Very impressive work, devs. System details: Nobara Linux, Kernel 6.6.7-200, Steam Deck OLED. Added the AppImage as a non-Steam game, switched to game mode and HDR was available as an option within Moonlight’s settings.

Hmmm… HDR is grayed out on my end…

matte-schwartz commented 6 months ago

This latest AppImage is a winner! Very impressive work, devs. System details: Nobara Linux, Kernel 6.6.7-200, Steam Deck OLED. Added the AppImage as a non-Steam game, switched to game mode and HDR was available as an option within Moonlight’s settings.

Hmmm… HDR is grayed out on my end…

Even on the build from an hour ago? The previous ones I tried hadn’t worked but this worked OOTB for me… happy to dump logs if it would be useful

johnyz333 commented 6 months ago

This latest AppImage is a winner! Very impressive work, devs. System details: Nobara Linux, Kernel 6.6.7-200, Steam Deck OLED. Added the AppImage as a non-Steam game, switched to game mode and HDR was available as an option within Moonlight’s settings.

Hmmm… HDR is grayed out on my end…

Even on the build from an hour ago? The previous ones I tried hadn’t worked but this worked OOTB for me… happy to dump logs if it would be useful

Can you link the build please…

cgutman commented 6 months ago

Here's a link to the working AppImage build with HDR support: https://ci.appveyor.com/project/cgutman/moonlight-qt/builds/48762923/job/8ciibh342qicq980/artifacts

chipgw commented 6 months ago

I can confirm latest build is working with Sunshine on Windows to host if you aren't getting it working here are some things to try:

what remains of the NVidia driver host did not work for me; upon starting a stream Moonlight would say the PC can't encode AV1 or H265 in HDR (my 30-series GPU can't encode AV1 at all, so that one at least makes total sense)
you need HDR turned on for the display you're streaming from in Windows display settings on the host machine; which yeah if Windows isn't outputting HDR then there's no HDR to stream.

dylanfrankcom commented 6 months ago

Thank you for your work on this! It looks like Appveyor has hit its download quota:

{"message":"Artifacts download limit (1024 MB/day) exceeded."}

Is there a mirror available?

matte-schwartz commented 6 months ago

Thank you for your work on this! It looks like Appveyor has hit its download quota:

{"message":"Artifacts download limit (1024 MB/day) exceeded."}

Is there a mirror available?

I anticipated this happening and uploaded a mirror here: https://pixeldrain.com/u/1zMvXLqi @ cgutman please feel free to delete if this is not okay

Moredread commented 6 months ago

I've tried the Appveyor AppImage on my OLED Steamdeck with a Nvidia 2080TI and Sunshine on my PC.

I can select HDR in Moonlight, but starting the stream produces a black screen.

When manually changing the codec to HVEC (AV1 encoding isn't supported by the graphics card) I get a warning "This PC's GPU doesn't support HEVC Main 10 decoding for HDR streaming" and the stream is shown in SDR (i.e. washed out colors and no HDR indicator in the brightness settings)

TheBacon00 commented 6 months ago

I've tried the Appveyor AppImage on my OLED Steamdeck with a Nvidia 2080TI and Sunshine on my PC.

I can select HDR in Moonlight, but starting the stream produces a black screen.

When manually changing the codec to HVEC (AV1 encoding isn't supported by the graphics card) I get a warning "This PC's GPU doesn't support HEVC Main 10 decoding for HDR streaming" and the stream is shown in SDR (i.e. washed out colors and no HDR indicator in the brightness settings)

Yeah this is my experience as well. Happy to provide any data that might be useful (unless the SD really can't do HEVC Main 10 decoding?).

johnyz333 commented 6 months ago

Had the same issues, set it to automatic, that worked for me.

zachrd commented 6 months ago

I also had this problem. Will check automatic in a moment.

arshiatn commented 6 months ago

"This pc's gpu doesn't support AV1 main10 decoing for HDR streaming" upon choosing AV1 "This pc's gpu doesn't support HEVC main10 decoing for HDR streaming" upon choosing H.265

I tried out 1080p, custom 16:10 resolutions (1200p and 800p). All gave me this message. HDR was on all the time.

Using a RTX 4090, Windows 11 and a LG C2 monitor.

johnyz333 commented 6 months ago

"This pc's gpu doesn't support AV1 main10 decoing for HDR streaming" upon choosing AV1 "This pc's gpu doesn't support HEVC main10 decoing for HDR streaming" upon choosing H.265

I tried out 1080p, custom 16:10 resolutions (1200p and 800p). All gave me this message. HDR was on all the time.

Using a RTX 4090, Windows 11 and a LG C2 monitor.

Try automatic, it should fix it

david-pili commented 6 months ago

Automatic results in a black screen

arshiatn commented 6 months ago

Automatic results in a black screen

You saved me writing an essay above. I tried out every decoding option beside "automatic". HDR works now. fabolous!

Edit: time to calibrate HDR on my Deck xd. It takes 1 minute :D

arshiatn commented 6 months ago

Automatic results in a black screen

It could be possible that it tries to give you AV1 and your GPU doesn't support it or you are using Geforce Experience? Before downloading Sunshine, I would get a black screen too upon streaming via Geforce Experiencing and selecting AV1 in my settings.

zachrd commented 6 months ago

Setting to Automatic results in a black screen for me. The Host is running Sunshine, on an RTX 4090. The Steam Deck is using the latest Preview Build of SteamOS (20231213.2). I deleted the config file for Moondeck and had the same issue.

cgutman commented 6 months ago

Hmm, that's odd that you'd get different results between Automatic and HEVC. I'll have to take a look at that. SDL has some quirks switching between OpenGL and Vulkan on the same window where it can get wedged into a state where Vulkan stops working, so it may be related to that.

For those users that are getting black screens with codec set to Automatic, can you provide:

Steam Deck model (OLED vs original)
SteamOS version
Resolution selected in Moonlight
Docked or undocked?
Display (internal display or external monitor)

Also if you press L1+R1+Select+X (or Ctrl+Alt+Shift+S on a keyboard) at the black screen, do you get the performance overlay?

Does streaming with HDR disabled on that same AppImage build work normally?

arshiatn commented 6 months ago

Automatic results in a black screen

You saved me writing an essay above. I tried out every decoding option beside "automatic". HDR works now. fabolous!

Edit: time to calibrate HDR on my Deck xd. It takes 1 minute :D

I calibrated my Deck OLED's Screen via "Windows HDR calibration" app and I had to use 2250-2300nits in second and third screen to completly remove the patterns in the middle. If you try to calibrate again, the patterns disappear at 800 nits (Where they originaly were for my LG C2). Really weird. Much higher than 1000nits on Deck. Is your software/deck itself using "Dynamic Tone Mapping on" instead of HGiG? Is there any option or any way to use HGiG on Deck?. On my LG C2 that supports max 800nits, I use HGiG and only see the patterns up to 800nits on second and third screen of "Windows HDR calibration"

Deck's Brightness was at 100%

Edit: I haven't seen this problem on Deck itself nor via Chiaki HDR. For Chiaki I calibrated my PS5's HDR settings for 1000 nits (0-15-15 steps from bottom in PS5's HDR calibration) and the deck was showing almost max1000 nits in the games I tested. On Deck itself, you could try out Cyberpunk 2077, Jusant or Ori Will of Wisps with following settings from my "calibrated settings for deck oled" reddit post (https://www.reddit.com/r/SteamDeck/comments/182uv9n/correc_hdr_settings_for_games_i_got/)

For Cyberpunk, calibrating my HDR to 1000-1050 nits in Windows gives me almost 400-500 max nits in Cyberpunk. Running natively on Deck itself, it hits almost 1000 and reaches a bit higher (1100-1300nits) for some light sources.

For Jusant for example, I am using same graphic settings like on my PC, same HDR settings (brightness max, UI brightness lowest) and I get almost 630 nits max in menu, 286 nits avg. Using Moonlight, I can reach almost max 630 nits when I calibrate my screen to 2600 but that fcks up everything else beside max nits, it ruins the whole picture. I tested out every parameter between 1500-2800 and they all look bad and not right. (Every parameter => +100 nits everytime) I believe Moonlight is using "DTM on" (Dynamic Tone Mapping On) and not HGiG. I didn't have this problem using ChiakiHDR or even on Deck itself so I find it really strange.

Ori, the best HDR implementation I have ever seen on PC, gives me a solid (almost) 1000 nits max HDR brightness and best HDR experience you can get out there while running it native on Deck. Using Moonlight, it looks too dark when windows is calibrated at 1000nits. You can hit almost the same max nits compared to native deck if you calibarte your Windows's HDR to 2600 nits. However everything else on the picture looks too overexposed and not correct.

Is there any option for you (@cgutman) to change the Tone Mapping at all? Is it possible to take HGiG instead of "Dynamic Tone Mapping"?

Edit: Via Chiaki I tested FF7 Remake, Cyberpunk and God of War. They all had almost the same HDR experience compared to running them native on the Deck.

Edit2: I forgot to mention that I was getting these nits on my Deck, Chiaki and Moonlight via HDR Waveform (Go to settings, display and scroll completly down. There you can change "none" to HDR Waveform.). It shows max, avg, min nits and much more.

Using W11, AutoHDR off/on (tested both in last 1-1.5 hours), reset my nvidia control panel and that changed nothing, using SteamOS 3.5.7 on the deck and using the AppImage that matte-schwartz uploaded.

Display mode Fullscreen, Windowed, Borderless windowed on Windows nor Moonlight change the HDR Brightness, as expected,

This porbably doesn't matter but changing the resolution, bitrate nor FPS didn't change the HDR brightness either.

Naadrius commented 6 months ago

The new build works like a charm on my Steam Deck OLED. HDR is working for sure!

The HDR Calibration clips at around 2250 nits for me as well and I can see that games usually don't get as bright as they would playing them natively on the Steam Deck. Not sure if this is fixable from moonlight.

Edit: after more testing it seems that most games on moonlight just cap out at around 690 nits according to steam deck HDR Waveform. On Hogwarts Legacy even setting HDR to 6000 nits did not do any clipping and HDR Waveform showed 693 Nits. Only way I get true 1000 Nits on Steam Deck Moonlight is when I open HDR calibration on PC. So there seems to be some kind of DTM (Dynamic Tone Mapping) in place.

Also I believe we have some black level raise with the current Moonlight build. My Steam Deck does not go full black when streaming Moonlight with HDR. You can test that on games with black loading screens or just pull up paint and full screen a full black image

alexandernilsson commented 6 months ago

For those users that are getting black screens with codec set to Automatic, can you provide:
Steam Deck model (OLED vs original)
SteamOS version
Resolution selected in Moonlight
Docked or undocked?
Display (internal display or external monitor)
Also if you press L1+R1+Select+X (or Ctrl+Alt+Shift+S on a keyboard) at the black screen, do you get the performance overlay?

Does streaming with HDR disabled on that same AppImage build work normally?

Tried the latest AppImage from the pixeldrain link. I can control the game on the PC via the deck gamepad, but screen is black when HDR is enabled.

OLED
SteamOS 3.5.11
Tried native and 1080p
Undocked
Internal display
Unable to get the Moonlight performance overlay showing with that gamepad combination or keyboard combination. The Steam performance overlay also doesn't show, screen is completely black.
Streaming with HDR disabled works as expected
Host: Win 11, 3090 RTX

FrogTheFrog commented 6 months ago

@cgutman the latest AppImage works for me and I've been trying to get flatpak to work. Alas, after frankensteining it from chiaki and your AppImage config, the HDR button is still greyed out :/. At this point I'll stop.

Here's the file in case you want to easily copy stuff: https://github.com/FrogTheFrog/com.moonlight_stream.Moonlight/blob/master/com.moonlight_stream.Moonlight.json

Rafalskyy commented 6 months ago

Maybe hdr surfaces isn't available in flatpak. See https://github.com/streetpea/chiaki4deck/issues/93#issuecomment-1830897848

FrogTheFrog commented 6 months ago

I've added it already :/. Will check the logs in the evening though

Moredread commented 6 months ago

Hmm, that's odd that you'd get different results between Automatic and HEVC. I'll have to take a look at that. SDL has some quirks switching between OpenGL and Vulkan on the same window where it can get wedged into a state where Vulkan stops working, so it may be related to that.

For those users that are getting black screens with codec set to Automatic, can you provide:
* Steam Deck model (OLED vs original)

* SteamOS version

* Resolution selected in Moonlight

* Docked or undocked?

* Display (internal display or external monitor)
Also if you press L1+R1+Select+X (or Ctrl+Alt+Shift+S on a keyboard) at the black screen, do you get the performance overlay?

Does streaming with HDR disabled on that same AppImage build work normally?

I remembered I switched to the beta channel for SteamOS. To rule that out, I switched to the current stable channel (SteamOS 3.5.7) and there it works!

The configuration that doesn't work, same as above but for the SteamOS version:

OLED
SteamOS 3.5.11 (Current beta SteamOS channel)
Tried several sizes, fullscreen/windowed and framerates
Undocked
Internal display
Button combination did not work on the black screen, but with SDR it works
SDR streaming (without the flag) works
Host: Windows 11, RTX 2080TI

FrogTheFrog commented 6 months ago

Maybe hdr surfaces isn't available in flatpak. See streetpea/chiaki4deck#93 (comment)

This is indeed the case! Ty for the info, I can stuff again...

arshiatn commented 6 months ago

Basically I just got the black screen people were describing above but a restart on my Steam Deck fixed it. I started Cyberpunk 2077 to see how its HDR compares to HDR in Moonlight and had the game open in background. I started Moonlight as second app on my Deck and connected to my pc. On my pc, I just wrote "cyberpunk" in search bar and entered it instead of going to steam then start it. Somehow Steam thought that I want to stream it from my deck, so it connected to my Deck and started streaming deck's screen to my pc. After closing it, my screen was black in Moonlight with "HDR on" on my pc. Restarting my pc didn't fix it but restarting the Deck fixed my problem. (SDR worked but not HDR)

.OLED .SteamOS 3.5.7 .1080p .Undocked .W11, RTX 4090

FrogTheFrog commented 6 months ago

I DID IT! Managed to enable that HDR button in my flatpak build!

@cgutman I copied stuff from chiaki and they have 2 patches for ffmpeg, FYI.

That aside, can anyone try this build out (as I don't actually have HDR host ready yet)? The installation is easy, just need to copy one-liner from readme in https://github.com/FrogTheFrog/com.moonlight_stream.Moonlight and execute it in the terminal on SteamDeck (will need sudo password).

bhoumik24 commented 6 months ago

I DID IT! Managed to enable that HDR button in my flatpak build!

@cgutman I copied stuff from chiaki and they have 2 patches for ffmpeg, FYI.

That aside, can anyone try this build out (as I don't actually have HDR host ready yet)?

The installation is easy, just need to copy one-liner from readme in https://github.com/FrogTheFrog/com.moonlight_stream.Moonlight and execute it in the terminal on SteamDeck (will need sudo password).

Enable HDR isn't greyed out anymore and I'm able to check the box, but it's not actually enabling HDR

FrogTheFrog commented 6 months ago

I DID IT! Managed to enable that HDR button in my flatpak build! @cgutman I copied stuff from chiaki and they have 2 patches for ffmpeg, FYI. That aside, can anyone try this build out (as I don't actually have HDR host ready yet)? The installation is easy, just need to copy one-liner from readme in https://github.com/FrogTheFrog/com.moonlight_stream.Moonlight and execute it in the terminal on SteamDeck (will need sudo password).

Enable HDR isn't greyed out anymore and I'm able to check the box, but it's not actually enabling HDR

Is there a way to visually check if the HDR is enabled without starting the stream? Or some other way without HDR host?

cianhorgan commented 6 months ago

That aside, can anyone try this build out (as I don't actually have HDR host ready yet)?

I can give it a go in the next hour, does the host system make a difference? I can do Nvidia or AMD GPU, both on windows unless I fire up a portable Linux image

moonlight-stream / moonlight-qt

HDR support for the Steam Deck #1117