Barrel/Pin-Cushion Distortion and Colour Seperation

[mungewell]

Seen it written (other) several places that for VR mode you don't need to distort, that is wrong. See attached images.

When view in headset the blue box appears true and the red is distorted inwards, the same as is seen on the social screen. So when the distortion of your output looks true on the social screen, you are about correct.

One thing that does confuse me is why a primary colour of red, green and blue is being split into other colours. Does this mean that the PSVR is not showing RGB pixels?

psvr_distortion.svg.zip

[mungewell]

Oh, it is not the RGB which is splitting, it is the white background. Switching to black make the lines stay together. D'oh!

[gusmanb]

Not sure where you read that, but for sure you need to distort the view, you need to apply two distortions, one to correct the spherical projection and other to correct the lenses cromatic aberration. For example, SweetFX/ReShade is used to do this and it has these parameters:

#ifndef VignetteToggle
//Toggle on/off vignette. Adds a soft vignette to the edge of the lense correction. [0|1] 0 = Off / 1 = On. DEFAULT = 1
#define VignetteToggle 1
#endif

#ifndef AspectRatioCorrection
//Toggle on/off aspect ratio correction for SBS stereo [0|1] 0 = Off / 1 = On. DEFAULT = 1
#define AspectRatioCorrection 0
#endif

//Scale entire lens size smaller or larger. Larger numbers = smaller screen size. DEFAULT = 1.0
#define ScaleFactor float(1.0)

//Seperation between lenses. X is first set of numbers Y is second set (no need to offset Y). Negative X number = further apart. DEFAULT (0.0,0.0)
#define StereoOffset float2(0.0,0.0)

//Chromatic Aberration. Change if your lenses requires correction. DK2 values = (0.99, -0.004, 1.014, 0). DEFAULT = (1.0, 0.0, 1.0, 0) = OFF.
#define ChromAbParam float4(1.0, 0.0, 1.0, 0)

//Distortion Scaling. Change if your lenses require different correction. A value of 0.55 = total coverage top to bottom but crops edges
//0.65 = better but with minor letterboxing. DEFAULT (0.65)
#define distortionScale float(0.81)

//Aspect ratio of each eye (for example 1920x1080 screen, each eye recieves 960x1080. 960/1080 = 0.8888888888888889 (or 0.9 rounded up). This also affects distortion too.
//A value of 1.0 seems to work best for distortion without warping. DEFAULT = (1.0)
#define AspectRatio float(0.89)

//Warp using standard DK2 distortion parameters. Best left as-is but could be tweaked if required. DEFAULT = (1,0.22,0.24,0)
#define HmdWarpParam float4(1,0.22,0.1200,0)

These parameters are posted by a group of people which is testing the PS VR with Skyrim and seems to work very well. (here is the link if you're interested, it's in spanish)

The graphics you posted are indeed very interesting as they will allow us to calculate the exact projection distortion, did you created it manually until it matched the values or got the parameters somewere?. If we create the same graphics but with only the blue colored as white in a black background and capture the result shown by the social screen we can also calculate the chromatic aberration parameters.

Will try to find some time to check it and find an information source on how to compute this with the help of these.

[mungewell]

Just used Inkscape (vector graphics program).

Drew square, put PVSR in VR mode, and then adjusted bezer points on the sides until they were roughly right. Cloned, flipped and positioned for the left eye.

[gusmanb]

Ok, will try to calculate the distortion angle from the SVG bezier path data.

[mungewell]

I don't think that you should trust my crappy drawings as a reference.... this guy does it properly. http://doc-ok.org/?p=1414

I've asked him about the PSVR.

[dproldan]

This is what you get when you use the RemotePlay app and the PS4 is in "VR" mode. Could be useful to make the corrections. http://imgur.com/a/8kz3Z

This is a checkerboard pattern taken from the remotePlay app. http://imgur.com/a/hTPZI

I don't know exactly when, but when the PS4 enters the "VR" mode, it starts applying barrel/chromatic corrections to the image and you can see it in RemotePlay.

Using this image, we should be able to find the perfect correction to apply. hope this helps.

[mungewell]

I made another image, might be easier to take measurements from. The extra white circle is approximately my limit of view (my IPD, eye relief, etc) psvr_distort2.svg.zip

[mungewell]

Some suggestions on settings for 'Tridef 3D': https://www.reddit.com/r/PSVR/comments/5ci74t/psvr_pc_games_hmd_barrel_distortion_tridef_only/

[gusmanb]

I've been all day with the VR player, I'm restructurating the code, cleaning namespaces, correcting geometries and projections and finally implementing the barrel distortion shader to correct the lens distortion.

This starts to look very promising.

Tomorrow I will check a procedure explained here to experimentally calculate the hmd values, I've tested the player with some params from cardboard and oculus dk1 and even if they don't look right the difference is really noticeable, tweaking these will give us the exact parameters. Whe I get the final params I will add a page to the wiki as these parameters are the key to configure tridef/reshader/sweetfx.

[gusmanb]

I've tested some values and got something very reasonable, not 100% perfect, the values need a bit more of tweaking, but the videos now look really good. Only a bit of distortion on the borders and only noticeable on large perfectly straight lines. Also there's still no chromatic aberration correction.

Here is the compiled player.

The player now will control the enter/exit from VR mode if the toolbox is opened, so I recommend it. Also, very important, configure the hmd screen mode to 120Hz, the difference is really huge.

There are a ton of controls mapped to keyboard, when I get time I will add a remapping utility and an interface for in-play control. Also, while on test, I need to add another key to dump all settings to share the values and tweak them.

left: go back 5 secs right: go forward 5 secs up: volume up down: volume down ctrl + left: modify video projcetion separation (some form of IPD) ctrl + right: modify video projection separation (some form of IPD) ctrl + up: change camera FOV ctrl + down: change camera FOV shift + left: go back 30 seconds shift + right: go forward 30 seconds space: pause/resume

these next are the keys to control the barrel distortion

q/w: change lens physical distance a/s: change baseline distance z/x: change FOV e/r: change screen distance to lens 1: enable/disable center indicator

There are some bugs I know, like the 360 mono mode needs a 45º rotation and things like these but nothing really important.

Before opening any video remember to select the video type, the frame type (side by side or top down) and the vertical FOV (this doesn't affects the projection, only limits the view range available with the mouse).

If some of you try this please report performance and quality.

[mungewell]

Hi, Don't actually have PSVR with me this weekend (supposed to be outside fixing my wife's jeep ;-), but tried the player with a couple of videos under Windows 10.

'Karate' from https://theta360.com/en/gallery/videos.html 'Drone' from http://www.360rize.com/vr/

Generally pretty looks good (just guessing on the distortion), however a few glitches: 1). Seams/Edge of video is a little off, see attached.

2). Resizes to Laptop screen which is <1080p. OK if only have external monitor active. 3). Side-by-Side option should be greyed out if video is mono. 4). Mouse pan limited to 360, should be infinite.

Can't wait to try these in PSVR. Good Job! Simon

PS. Is the intent to bundle the Viewer with the Toolbox? Will it work on Linux?

[gusmanb]

Hi Simon.

Let me answer:

1-This is correct, the applied distortion for lens correcting may get some parts inside/outside the screen but anyway I'm correcting a missing part on the shader, may be this is related. 2-The window is fixed to 1080, very strange it resizes to the screen res, are you sure it's not the reverse, it changes to 1080 but as the screen is lower than 1080 it's being scaled by the screen? 3-Yes, the UI is minimal to say something, when it gets stable I need to add a lot of things to it. 5-Yes, that's correct.

I will post more info if I get the shader corrected.

About linux, not 100% sure, I'm using C# + managec C++ + pure C, the C# and C part can be done in a portable fashion way, but the managed C++ code not. I can skip the managed C++ part using P/Invokes but that will be in a future as this complicates the developmen (with managed C++ you can have mixed code, managed an unmanaged, so the C++ is a wrapped between the manged UI and the native player code). Also I'm using libvlc to decode the video and not sure how much portable is it, it should work but it's untested.

May be if I sepparate the UI from the service like is explained on the UI issue can help to this, I'm really thinking about using Qt as it's C++ and would have no problem with dll invokes and can be cross platform, but I never used it so not sure for now. Do you recommend any language/framework for cross-platform UI? My day to day development for linux is based in services so I never bothered on finding a really good UI cross-platform solution and always used .net as for services works really well, any suggestion is welcome.

[gusmanb]

I'm really excited, I've corrected the shader and computed some of the params and now things look really well.

I have computed these parameters:

68.0f -> Vertical FOV, derived from specs, a 100º HFOV is translated to 60º VFOV for a 16:9 aspect ratio 0.0630999878f ->Distance between the lens centers 0.0394899882f, -> Distance between viewer baseline and lens center

These three parameters are correct (may be a bit of precission loss because of floats, but exact in practice).

To calculate it I have added a test pattern as you can see on the screenshot, then the shader has an option to enable a dot on the projection's center, so the procedure has been easy, tweak the params at real time until the center dot matches exactly the test pattern center and comparing each eye view individually there's no displacement horizontally nor vertically and straight lines crossing the centers are completely straight.

We are missing only three more parameters, screen to lens distance , K1 and K2 (distortion coefficients) and for these I have no way to calculate them empirically, screen to lens affect the distortion projection and K1/K2 affect the same, so I have no idea on how to check these unless we know one of the others.

I'm going to use K1 and K2 from the oculus rift DK1 as these seem to be similar to the PSVR and check the distance.

[gusmanb]

Ehm... not sure if they're correct, but using the DK1 params (0.22f & 0.24f) and a screen to lens distance of 0.0354 the distortion is completely gone, perfect straight lines on the test pattern.

Also, now the videos are comfortable to watch, with previous settings some kind of tension was noticed, but with these the eyes are completely relaxed, no force noticed at all.

I'm going to create a wiki entry with these and upload a new test player for testings.

Now only the sensor integration is missing to achieve a complete VR player. Yay!!

[gusmanb]

I've uploaded the new player test here: test_player_2.rar.

All the test have been positive, I've compared the result of a 360º mono video on PS4 and PC and they look exactly the same (except for chormatic aberration :P), no distoriton, same FOV, etc etc.

[gusmanb]

Aaaaaand the chromatic aberration is ready :D

I will upload it tomorrow for tests.

[mungewell]

Confirmed the video in the headset looks good, and is sent to the correct desktop - maybe I had duplicate on both outputs set before.

The panning with the mouse is a little sick-inducing... I've seen some applications reduce the image to just central circle of the view whilst panning (other than with headset), maybe this is an idea to implement. Very excited to try out Sensor fusion when it works.

[Wallbraker]

@gusmanb I'm sorry to necro this thread, but do you happen to have the shaders you used for distortion laying around?

[gusmanb]

@Wallbraker All the shaders are in the engine.h header in the VR video player.

https://github.com/gusmanb/PSVRFramework/blob/master/VRVideoPlayer/engine.h

[Wallbraker]

@gusmanb Thank you very much!

I guess these are the values you are using? https://github.com/gusmanb/PSVRFramework/blob/master/VRVideoPlayer/vrdevice.h#L102-L125 and the code for dealing with the different matricies are all in https://github.com/gusmanb/PSVRFramework/blob/master/VRVideoPlayer/vrdevice.cpp right?

I'm asking because I'm working on getting the distortion as good as possible for the PSVR driver in Monado. We are getting close to have 6DOF tracking work as well.

[gusmanb]

@Wallbraker Yep, that's the code. You can see how the parameters are applied to the shaders in engine.cpp.

[Wallbraker]

Yupp spotted that, thank you very much. Hoping to have something inspired by that next week that will hopefully make the PSVR not give me distortion sickness :)