alvr-org / ALVR

Stream VR games from your PC to your headset via Wi-Fi
MIT License
5.24k stars 473 forks source link

Quest brightness/gamma value incorrect #71

Closed Kuuko-fokkusugaru closed 4 years ago

Kuuko-fokkusugaru commented 4 years ago

So I have noticed something few days ago. I played and still own CV1 but used to play mostly Beat Saber while Quest through PC with ALVR it's used mostly for VRChat. I didn't play VRChat much with CV1 so I didn't notice this before. I was playing with my boyfriend and we were in a very dark world called "Submerge". There is a part where you are supossed to see a creature on the deepest dark. While he could see it fine on the CV1, it was mostly pitch black to me. It was really impossible for me to see such details. I did further tests and the difference it's HUGE! I am not sure how I didn't notice this before. CV1 image it's way way brigther than Quest one. If I take screenshots with both, the image it's the same (obviously) but it's on the headset side where the image reach differently. My guess is, since we are using Rift S as detection method, a different color profile it's being applied. Rift S uses LCD screen with less black levels which isn't weird that the image being sent to the headset may be darker to compensate for the lack of pure blacks. This on Quest makes a huge difference as the image it's extremely black and details are missing. Or maybe we should apply certain brightness/gamma value on the client side to display the image correctly. I have no idea wether the image sent it's the same that we see on PC just displayed different on CV1 or if it's sent directly with such brightness. I have attached an image of how the same map looks in Quest vs CV1. The image is not real, I just altered it in Photoshop but I made it pretty close to the real ones (despite of how it may look, I didn't exagerated at all) and was taken on a map called "The Mansion" in VRChat in case that someone want to give it a try. As you can see, the map it's so dark that you can't see the vegetation and inside of the building it's impossible to see some of the rooms.

I am not sure if this could be fixed or if could be some workaround. We are using Rift S profile but we set a custom resolution and refresh rate, right?. Could we use CV1 profile for the headset in the same manner and leave the S touch controllers? It seems that S touch controllers are compatible with CV1 after all. Maybe this could fix the issue with the image. You really don't tell the difference untill you play some games on a different headset and notice the huge amount of details that you are missing. Dark caves in Skyrim it's a good example too.

I want to add that on PC the game always looks this dark. I have noticed a similar behavior before where years ago, some games wouldn't be recorded with it's right gamma value which made the games looks brigther while playing but way darker on the recorded video/screenshots.

OculusScreenshot1573354015

ghost commented 4 years ago

This would explain why some maps feel uncomfortably dark. I literally could find my way around some maps with night environment.

JackD83 commented 4 years ago

Should be possible using a shader

zarik5 commented 4 years ago

I can do this

Kuuko-fokkusugaru commented 4 years ago

@zarik5 awesome! Do you guys think that there is a way of knowing the exact values? My guess is that may have to do with the gamma displayed in the headset itself even tho, some recorded videos can't be altered to get back some of the details displayed on CV1. It seems that the image it's so dark that these pixels are set as complete black so you can't recover back what were supossed to be behind those black pixels (I would need to further test this anyway). My point is, it's there any real info regarding color profiles for each headset or similar? Because Vive it's kinda more open source so maybe that kind of information it's of easier access. And since Vive also had similar screens, we could use the same if were the case.

Either way, can't wait to see some progress on this ☺.

zarik5 commented 4 years ago

I would also like to add control of brightness, contrast and saturation. @KitsuneShan do you know what is the best order to apply these effects, if there is one?

Kuuko-fokkusugaru commented 4 years ago

@zarik5 not programmer but I guess you would apply brightness first, then contrast for the washed out black and whites and then saturation to fine tune colors. But I feel the main thing should be gamma. It kinda feels like when you have to use sRGB for proper gamma values. So maybe all goes down to the gamma value that the client launch with? As I suggested before, we should first find wether each headset uses indeed it's own color profile and wether this is decided on SteamVR side or headset directly. If we figure that, we can know for sure wether it's that or just different kind of calibration for the screens. I'll try to create a world for VRChat to compare different shades and try to deploy for pc and Quest. Then we can compare if looks the same for both versions which would mean that Quest screen it's just differently calibrated. Of course, to have gamma, brightness, contrast and saturation controls would still be awesome. Imagine being able to load color profile through LUT images. Then we could adjust easily just adjusting in Photoshop or similar and saving the LUT withing ALVR exe :smile_cat:.

zarik5 commented 4 years ago

Implemented gamma, brightness, contrast and saturation correction in #74. There is no automatic color profile detection, you have to set the values yourself. @KitsuneShan when the PR will be merged I suggest you to try a gamma value of 2.2.

HolyCaonima commented 4 years ago

I guess, the video stream is some how being degammaed. A gamma value of 2.2 may help.

HolyCaonima commented 4 years ago

Maybe some client side bug, I guess. Using a SRGB Render target, or something like sqrt(x), pow(x, 1/2.2) in shader may cause this.

Kuuko-fokkusugaru commented 4 years ago

@HolyCaonima I don't think that it's exactly a bug. We are just displaying the video as is, in the same way that it's displayed in desktop. So the game displays different gamma values while it really should apply the same gamma on desktop too. So I'm not sure if all games are displayed with the same gamma or each uses it's own. On Quest it's always the same app so the gamma it's fixed. We will see if then other games would get double gamma and be too bright. I may test with more games.

zarik5 commented 4 years ago

I tried VRChat but I didn't see any problem with gamma. Are there any other games with a wrong gamma?

@KitsuneShan have you tried to compare the same VRChat maps between the pc and the native Quest versions?

Kuuko-fokkusugaru commented 4 years ago

@zarik5 the problem is not with VRChat only, it's with every single game, even SteamVR itself. After some tests I can confirm that we are using a 1.0 gamma while CV1 and possibly the rest of headsets are using a 2.2 value.

I have compared different VRChat maps and yes, there is a huge difference when playing the native version of Quest vs PC with ALVR. If we compare the native version, everything looks brighter just like it does in CV1 while with ALVR everything looks pretty dark.

I just can't test every single game because most of them are on Oculus store for CV1 and I only have a couple of them on Steam. My only concern it's not knowing wether the values are wrong or not because there is no doubt about it after the tests, but to know wether some games override the gamma values or not. But I really think that everything may look perfectly fine considering that the whole SteamVR runs way darker than it should.

To make it easy, you can test it by yourself. I have made a simple black levels chart that let you test and see the different levels through the SteamVR background as a 360 image. There are 4 charts, all the same, just at different distances. When using ALVR, you can see the background pretty much black, can't see any difference from the square charts and you can see only to number 007 and extremely faded 006. These colors represent the RGB value so 007 means 007 for all the RGB channels. On CV1 you can see every single number, even the first one that it's 001 and it's over a completely black background. You can even see the background with a different tone from the square charts (background is 001,001,001 while charts background is 000,000,000).

Here is a link with the SteamVR 360 panorama if you want to test it by yourself: https://steamcommunity.com/sharedfiles/filedetails/?id=1918669376

Here is another comparison. This time it's 100% accurate as I used the exact values. So, as it is now, ALVR should have a 2.2 gamma on the client side activated by default. But just in case, we should be able to turn it off too just in case that some games would apply their own gamma values overriding SteamVR one (which I highly doubt it would be the case but since I can't confirm this at 100% then it's better to be safe). gamma comparison

Edit: It would be interesting knowing how TPCast deals with this. Maybe it's just at 2.2 gamma always. But I can confirm that, Oculus Home on Quest, for example, runs at 1.0 gamma since I tried to open the 360 image on the gallery and had the same issue of not seeing all grey tones properly. But I guess this is understandable, as images and videos may be displayed just like on any desktop with 1.0 gamma.

JackD83 commented 4 years ago

Hold on, do you perceive it as darker or is it actual darker? Do screenshots you make in the cv1 and on the quest look different? The both have different panels which will look different. There is no general gamma problem on the quest with alvr. Steamvr with gamma 2.2 is waaaay to bright

Kuuko-fokkusugaru commented 4 years ago

Hold on, do you perceive it as darker or is it actual darker? Do screenshots you make in the cv1 and on the quest look different? The both have different panels which will look different. There is no general gamma problem on the quest with alvr. Steamvr with gamma 2.2 is waaaay to bright

Well, let's take into account that depending on the kind of monitor and how you have them calibrated, the image shown may vary but yes, it's perceived exactly like that. But as side note, CV1 with 2.2 gamma may not look that bright if you compare it with a monitor. I guess the CV1 doesn't use a so extreme "backlight" to not burn your eyes but regarding contrast it's exactly as I described and shown. Like there is literally no way that CV1 isn't using 2.2 gamma because, no matter how different the panels are, you can't get anything closer to CV1 without 2.2 gamma even if you increase brightness or contrast to the max on your monitor because the range of grays it's just completely off. With 2.2 gamma you get more range of whites and less of blacks which is what happens with CV1 and probably every single headset except Quest just because ALVR it's set at 1.0 and the games on desktop always seems to play at 1.0 too. If you think that Steam looks too bright at 2.2 may be because you just got used at seeing it at 1.0. I didn't noticed this either till I used the default background for Steam on both headsets and returned back to CV1 for the tests after so many time playing only with ALVR. The difference it's huge. And you can see it clearly if you compare dark worlds inside of VRChat from native mobile version vs ALVR. I just didn't want to use more VRChat examples to not make it look like it could be a single game related issue so I instead made that one Steam background where you can test by yourself. And these pics aren't taken from the headsets itself, they are manually calibrated changing gamma from 1.0 to 2.2 since it's the only way of seeing them exactly as they are displayed. So, yeah, unless I have an issue on my computer (which I doubt as this is more related to ALVR app), the background on ALVR only shows max to 007 being 006 extremely fades but slightly visible if you look at the big one for a while but the top line it's impossible to read while on CV1 it's extremely clear just like in the example image. I'd recommend to use that Steam background and try. I could also use it on a VRChat world and deploy for android too but, as I said, I don't want to make it look like a VRChat only issue. It happens with other games too. So yep, there is a gamma problem with ALVR app being at 1.0 rather than 2.2 which seems default for all headsets. I'm not sure if this is different using different devices for ALVR besides Quest. If someone uses a smartphone or something else for ALVR, they could give it a try too.

zarik5 commented 4 years ago

When debugging the color correction shader I was using scrcpy and if I turned brightness up to 1 (completely white) I can see that the white looks muddy, but when I put the headset on it looked alright (the stream from the pc instead is completely white). On the Quest home and other games, on scrcpy, patches of white have the same muddy tint. I think this is either a problem with scrcpy or more likely a color profile that the Quest applies system wide. When I was debugging the FFR shaders I remember seeing true whites on scrcpy, so I think this color problem was introduced with a firmware update. I also checked that I had the night light switch turned off when testing.

Kuuko-fokkusugaru commented 4 years ago

Well, I can't confirm that but I have been using ALVR since long and the gamma issue has been always there. I don't think it's a bug either, I simply think it's just wrong set at 1.0. Because, technically, who would want to see videos with 2.2 gamma after all? But when we talk about games I guess the thing changes. In the case of games like VRChat, the problem propagates since it has desktop mode too which defiintely should use a 2.2 gamma as well or desktop users will have a hard time playing some of the darkest worlds but maybe this can't be done unless they go into full screen? All this gamma stuff it's a bit weird after all lol. For example, pics taken on CV1 still shows a gamma of 1.0 as expected because the games itself has a 1.0 gamma and it's just the output video who is displayed with the correct one. In our case, we are simply displaying a video and I guess no one took into account that gamma would be wrong. If you didn't use any other headset before, this is extremely hard to guess and one may simply think that a smartphone screen is not the same as a VR headset. But both, CV1 and Quest uses pretty much the same screen, same type just different resolution and Quest one being limited to 72Hz through hardware (tho Oculus confirmed that the screen could be easily set at 90 or even higher through firmware change).

Couldn't we just set the ALVR apk to display at 2.2 gamma and make a quick test build? Or maybe it's more complicated than I think?

Btw @zarik5 , whites shouldn't get affected by this. I mean, completely white. Complete black and complete white should display fine, gamma only affects the range in between. I am not sure about the issue that you mention, but it's also hard to compare against a monitor if isn't extremely well calibrated. In my case, both of my monitors are professional for designers, one being BenQ with a VA panel wich shows higher contrasts than Quest at 1.0 gamma itself and the other a Color EIZO that it's IPS and have one of the widest color ranges among 10 bits color. And even tho the difference with something running at 2.2 gamma it's extreme. In fact, knowing that Quest uses OLED, we should see the image even way better than on a monitor even if it's at 1.0 gamma due the higher contrast yet it's still extremely far from CV1. I can also confirm that, after researching a bit, Vive uses a 2.2 gamma as well. I just guess that TPCast itself sets the Vive at 2.2 gamma when streaming games.

Another good test could be using Oculus Link but the cable isn't out yet and the update it's too recent for me to research a cable alternative here in the EU. But I guess that would be the definitive proof that we are on a wrong gamma (unless Facebook has done the same mistake lol since the image it's also streamed as a compressed video...).

zarik5 commented 4 years ago

I understand that whites shouldn't change if you change gamma. By the way, displaying your test image on my smartphone (which has an OLED screen) I can easily make the ALVR side look like the the CV1 side by increasing the brightness and viceversa by lowering the brightness. Unfortunately AFAIK there is no built in brightness adjustment on Quest and with the color correction I implemented there is inevitably white clipping. If you want to do some tests you can compare the pc version with ALVR and Quest version of VRChat and other games, then post some screenshots.

zarik5 commented 4 years ago

@JackD83 if the master is stable you can make a new release, let it be just because I see people online that thinks ALVR is dead...

Kuuko-fokkusugaru commented 4 years ago

@zarik5 yeah there is always ways of getting something close to the example shown but believe me it all goes down to the gamma value and wether we display RGB or sRGB. Being professional 3D designer I had to deal with gamma a lot between different render engines, images with gamma profiles, etc... it's a big headache but I could learn some about it.

As I said before, I already did test with VRChat maps and compared them between CV1, ALVR and the native android version for Quest and I can already state the huge difference in gamma. Basically, CV1 and mobile version looks the same (besides the map differences itself between platforms) while the one with ALVR looks extremely dark. This isn't specially noticiable if the maps arent really big. Even those with mid tones may look fine and it's something that you just get used to it. But some of the darkest maps like mazes and the one I mentioned on the OP, are maps that are pretty hard to play or unplayable all together. Yeah, probably, on a perfect world, those maps could have been designed with desktop and VR players in mind and made them not so extremely dark but we can also guess that VRChat devs itself could have added proper gamma for the desktop version. But it's a game focussed to VR after all.

I wonder if anyone tested the SteamVR background link that I shared above because it feels like no one tested theirself lol. Because I did, in two computers, CV1 and ALVR and well, I already posted how it looks on them. I am now on the work of a simple world for VRChat but I am new to this and don't know Unity much so it's a bit cumbersome ><.

zarik5 commented 4 years ago

Ok, I tried the background and indeed I can barely see the 9. This is the Quest capture on scrcpy: steavr bg

This is a snapshot on the debug video: vlcsnap-2019-11-23-17h46m32s012

Both look exactly how I see with the headset on.

zarik5 commented 4 years ago

This is with gamma 2.2 (actually I hardcoded the texture format to be RGB, using my shader would probably cut away the 002 because of color aliasing): vlcsnap-2019-11-23-18h09m09s994

I can see this is not right (other than the trees being too bright) because there are too few dark numbers and too many bright ones. Using the default settings gives a more balanced gradient between the numbers.

Kuuko-fokkusugaru commented 4 years ago

Do not worry about too few dark numbers, it is how it has to be, it's the point of gamma at 2.2 and it's how it looks on CV1 and Vive (talking about the background image.

About the trees, I am not sure. It may not be the same to bump the image to 2.2 gamma than just letting the headset go into 2.2 mode display while playing a 1.0 image/video. If you bump the brightness changing the gamma through a shader on a 8bits video, you may be running out of colors quickly as you are clamping some values probably. But if the video remains at 1.0 and the headset changes to 2.2 while displaying it, the video should remain with a big range of colors as the video itself isn't being modified at all. It's similar to working on linear vs non-linear textures. But we are talking about video so I am not 100% sure here as is not my main area. It's there a way of storing gamma on videos just like we can with PNG files among others less common formats? Could we then display the image properly?

I may add that, despite of what some people thinks, the background test image on CV1 it's even lighter than the screenshot that you took yet not everything looks overbright. I really wish I could take some screenshot directly from the headset lol. I wonder if I can use a camera right inside of the lens so you guys can see and compare by yourself XD.

Kuuko-fokkusugaru commented 4 years ago

@zarik5 it's possible to set the texture mode to sRGB instead? If I set my background as sRGB on different applications, it's displayed as it should with 2.2 gamma. So maybe set it to sRGB and not add any correction to the image? That should, technically, work unless there is some difference between setting the texture to sRGB vs setting the Quest screen to sRGB through the apk.

zarik5 commented 4 years ago

it's possible to set the texture mode to sRGB instead?

Most of the time, including in the home, SteamVR wants sRGB textures. ALVR just does what SteamVR wants. I forced RGB only with the last image I posted.

I may add that, despite of what some people thinks, the background test image on CV1 it's even lighter than the screenshot that you took yet not everything looks overbright

I reiterate, maybe it's just brighness, or maybe also the different response of the screens.

@KitsuneShan for now I don't know what else I can do. Just wait for the next release and see if you can fix the color with the handles I made.

Kuuko-fokkusugaru commented 4 years ago

@zarik5 oh I didn't mean the textures on SteamVR, we can't change that. I am talking about the texture being displayed on ALVR client, the video itself which I guess it's displayed as a texture.

And I guess it may be impossible to convince you :stuck_out_tongue: . Despite of what you may think, I am 100% sure of the gamma values being used. And I already confirmed that CV1 as well as Vive uses a 2.2 gamma for their screens. Even Quest itself does. But this is controlled through applications so just launching ALVR on Quest wont make it 2.2 if isn't programmed for that I guess. Same reason why the gallery, for example, displays the images with 1.0 gamma. This is understandable as I pointed before. Yet games are still set at 2.2. I really wish more people with different headsets could comment here but I guess we wont find many that invests time in ALVR if they already have a different headset like I do. But only then I guess you guys would believe me. I will try to set my camera to a fixed exposure and take some pics directly inside of the headset. I am also almost done with the VRChat map which isn't much different of using the background on SteamVR but if that makes it easier to convince you guys then it's fine :stuck_out_tongue: .

zarik5 commented 4 years ago

You don't have to convice me, I belive you 🙂. It's just that it's a subjective matter (I like it more as it is now) and I don't see any evident bug in the code to fix. If anyone else has a similar problem is welcome to comment.

Kuuko-fokkusugaru commented 4 years ago

Yeah because isn't really a bug and more something like we just missed. But we should take into account that games are meant to be played with a 2.2 gamma value on every headset. Playing at 1.0, even if more plesant for some, it's not the correct way on which the game was made. That also means that you lose a lot of details on dark areas and not just on VRChat which may be the less important game regarding this (you can always skip dark worlds) but games like Skyrim leave you pretty much blind on dark caves. Minecraft with Vivecraft it's another example. Entering on a cave it's pitch black while it should not. Even on console and mobile, you can see a bit in the dark even if it's extremely difficult due the lack of light. With ALVR you can barely reach the entrance and everything it's black and when using torch you can't see too much a few meters from them as if they are extremely low in bright. And that's just to name a couple of games. Of course, some others games may suffer less if there isn't much darkness. I didn't try but I bet Elite Dangerous may be a huge pain to play too.

JackD83 commented 4 years ago

I will prepare the next release and everyone can fiddle with it

Edit: Here it is ev8

TC-VR commented 4 years ago

I had noticed that there was indeed a difference between Oculus Link and ALVR brigtness/gamma. I've used the Apollo 11 HD Moon arrival scene to compare and by adjusting the Gamma to 1.05 i'm getting great results. The link is a lot brighter, even to bright to my liking, reminds me of my Rift S blacks and colours somehow...

Link to Youtube video: https://youtu.be/XYW0EtOKq2w

Kuuko-fokkusugaru commented 4 years ago

@TC-VR Awesome video and interesting results. It seems that Oculus itself it's using proper calibration when conencted with the Link cable. To be honest I wasn't expecting them to take all this into account.

Have you tried the SteamVR background that I did for the test? Could you please use it and take a couple of screenshot comparisons? I am not sure if the screnshots taken from Quest would show the difference in gamma tho but if it shows in the video then it should on screenshots too. On CV1 case, even if I take screenshots, they will be taken directly from the game so you wont see any difference at all.

BTW, how does SteamVR detects Quest? As Quest? Rift S? Generic device??

TC-VR commented 4 years ago

@TC-VR Awesome video and interesting results. It seems that Oculus itself it's using proper calibration when conencted with the Link cable. To be honest I wasn't expecting them to take all this into account.

Indeed, when playing Vader Immortal using the Link, i get the same clarity/brightness as the native Quest portal. When using Revive to play Vader Immortal using ALVR, it's very dark... I quess you are right that the Quest 'Rift S' colour profile is allready implemented in the Oculus desktop software but not in Steam VR, which would explain the brightness issue using Revive (through Steam VR).

Have you tried the SteamVR background that I did for the test?

No, will do tonight.

BTW, how does SteamVR detects Quest? As Quest? Rift S? Generic device??

Via Link: It's detected as a Rift S Via ALVR: It's detected as a Rift S as well

Kuuko-fokkusugaru commented 4 years ago

@TC-VR damn, that doesn't help much lol. Though it would be detected as Quest itself. So I guess it's the Oculus app inside of the headset itself applying proper color correction and gamma. It would be awesome if we could access to those parameters from another app running in background. Something that would dump the calibration of the screen to some log files or similar. Then we could know what's going on and get the exact calibration for ALVR. To be honest I don't see myself giving up on ALVR after getting the Link cable lol. Wireless VR it's too good.

JackD83 commented 4 years ago

Can I close this issue or do we need something else?

TC-VR commented 4 years ago

You can close this issue, since using the gamma on 1.10, it's working with near perfect clarity and contrast. Thank you

Kuuko-fokkusugaru commented 4 years ago

Yeah feel free to close it. It would be nice if we had the right values to make it as close as the rest of headsets but we can tweak everything manually anyway. Thank you guys for your hard work, very appreciated.