A strange wobble problem

[HelenXR]

Problems that have been bothering for months: HMD has access to the steamVR, in use, rotate in the horizontal direction of 360 degrees in the process, always find a special point, this point is near the screen jitter (see "shake.mp4", then HMD, standing on the table), in addition to other locations outside of this point, is normal (see the "normal.mp4"). I thought it was HMD IMU interference caused by abnormal, I put HMD rotation into Yaw, Pitch, Roll found that they are very small changes (see "10_seconds_sjake_hmd_rotation_information.txt"), what is the problem? Video and data. Thanks for any help

[TheDeveloperGuy]

The point is probably zero radians and could be a miscalculation of angular velocity when the orientation moves past that point (ie. a problem with your hardware or driver).

[HelenXR]

@TheDeveloperGuy Thanks for you replay YPR(137.595,1.39623,-0.462021) is in degree as you say,roll and pitch maybe very close to zero(radians),What's the situation? Why is there a problem near 0?my drivers' vecAngularVelocity and vecAngularAcceleration always zero.

[HelenXR]

"zero radians" maybe is not issue.because I manually modify the rotation from (137.595,1.39623,-0.462021) to (137.595,21.39623,-20.462021) the shake exist,however I modify the rotation from (137.595,1.39623,-0.462021) to (147.595,1.39623,-0.462021) the shake is missing.

[TheDeveloperGuy]

@HelenXR If you are not extrapolating angular velocity then my observation may not be suitable for you. In any case, the problem may still be something in either your hardware or driver.

[TheWhiteAmbit]

I observed behavior like this also. I think it maybe has to to with a kind of position prediction timing in the OpenVR api gone a bit wrong. My observation is that this happens when rendering at a critical framerate. The timing prediction (at least the values you get from the api) has sometimes an odd jittering of timing values. If you assume these are the base for the position prediction it may be the cause. Also I observe this only periodically. I don't have much hope on this, since my but report on this is untouched for a long time now...

[TheWhiteAmbit]

https://github.com/ValveSoftware/openvr/issues/72

[LoSealL]

@HelenXR If you are developing your driver based on IVRDriverDirectModeComponent, DO NOT rely on the pPose in SubmitLayers call

[HelenXR]

@TheDeveloperGuy I can add yaw degree by keyboard,maybe in driver,But I can't find out where the problem is~

[HelenXR]

@LoSealL My driver not base on IVRDriverDirectModeComponent , only modify EDID & whiteList to support direct mode

[echuber2]

@HelenXR I couldn't understand the full details of what you're doing here, but if you are calculating your own prediction for the IMU, could the issue be related to prediction and reprojection settings? Are you smoothing or filtering the poses at all? If the issue is related to interaction with reprojection, then the problem may present when the framerate is naturally lower during a more intensive view. I got some useful information from Aaron Leiby about this in #518 including info about the prediction modes and interpreting the frame timing.

@TheWhiteAmbit I would suggest the same possibility for issue #72

[LoSealL]

@HelenXR So, are you monitoring your FPS, does it keep at 90Hz?

[HelenXR]

@echuber2 I don't prrediction and reprojection settings. pose data is Stable when the screen is shake(Video and data can see this).

[HelenXR]

@LoSealL my panel is only 60FPS.

[LoSealL]

@HelenXR It doesn't matter what your panel's refresh rate really is. You can keep an eye on your application's real time fps to see if there is a drop when judder occurs.

[echuber2]

@HelenXR What about prediction and reprojection settings? Sorry, I don't understand what you typed.

My point is that although the pose seems mostly stable to the user, there are slight head bob movements that are natural and unavoidable. Overactive prediction settings, or a disconnect between the prediction algorithm and the reprojection algorithm, can cause the judder you are seeing just like this. In fact I conducted an experiment this year where we messed with the settings on purpose and saw artifacts just like that one.

[echuber2]

If you mean the HMD is only 60FPS or whatever, I'm not sure how SteamVR is applying reprojection to that. But you can watch the SteamVR monitor window to look for dropped frames (in red). You can also get the console window on the desktop to spam the actual HMD framerate, regardless of whatever the monitor's own framerate is.

[echuber2]

Another possibility I can imagine is that if there is any natural window light coming into the room from just one direction, the HMD sensors might react strangely in some orientations and not others. You could test that just by rotating the origin in the virtual space, or teleporting.

[HelenXR]

@LoSealL the frame is stable,I can see this in steamVR--->setting "Framerate Good".

[HelenXR]

@echuber2 at first,thank you for your patient reply about "not prediction setting" is that vecVelocity &vecAcceleration always set zero in pose.

the strange wobble point is missing when i do recenter, but i don't know why.

[echuber2]

What tracked devices are you using, and are you using GetDeviceToAbsoluteTrackingPose() at all?

[HelenXR]

hmd is custom. GetDeviceToAbsoluteTrackingPose() interface is openvr client api. I custom my HMD only develop in openvr driver api.

[echuber2]

Well due to my lack of understanding about the driver API, I'm not sure I have a solution. But try this: Open the SteamVR settings and turn off everything related to reprojection. (no async, no interleaved, no "always-on"). Does this help the situation at all?

[HelenXR]

I set this as you say"no async,no interleaved" like below:

shake still exist, "no always-on" mean "steamvr status window always on top"?

[LoSealL]

@HelenXR

Framerate Good

IS a fake monitor.... just log your fps in console or whatever, I can only imagine that the judder occurs because your ATWed pose is not the correlated rendered pose, which will cause a wrong reprojection. Even if you turn off reprojection, the non-stable FPS still cause judder because the picture is not updated continuously.

[echuber2]

@HelenXR Under Developer there should be another checkbox saying "Allow always-on reprojection" or something; make sure that is unchecked. If this doesn't improve things then it looks like the prediction algorithm isn't the issue.

I agree with LoSealL that you should log your actual FPS in the console window to see what it is exactly. If indeed your FPS is dipping noticeably during the "juddery" scene, then you may actually need to turn on all of the reprojection options to make it look stable.

tldr: If reprojection is working correctly, it should make the image more stable than default. If it's not working correctly, it will make the image look worse than default.

If none of this helps, then someone may need to inspect your code.

[echuber2]

Also, if you have Nvidia, go into the Nvidia driver options and make sure that vsync is turned off globally. You can also make a profile for your application exe and force vsync off there, and also turn off triple buffering. The Nvidia driver vsync mode isn't necessary to sync the HMD, and it actually messes up SteamVR. This is something that caused major frame drops for me.

[echuber2]

...and in the SteamVR options, you should also try disabling the supersampling AA option to get better performance.

[HelenXR]

Thank you all for your patient response, but I think the problem may not be on Reproject, because: In the InCell VR game, in the Yaw direction of rotation, can find the jitter point, at the time the Yaw angle of arrival about -157 degrees, and only at this point will appear jitter. At this time, I perform one of below ways as follows after the disappearance of jitter:

1. to maintain the state of jitter(HMD maintain state also), I open the steamVR Home application, the jitter missing.
2. when the InCell VR jitter occurs(HMD maintain state also), once I execute a recenter, the jitter missing.

[LoSealL]

@HelenXR All those two methods change the view right? The picture you see in hmd is different from jitter's one?

[HelenXR]

All those two methods change the view right? ---yes The picture you see in hmd is different from jitter's one? ---which picture?

[echuber2]

Reposition your objects in the game world to see if it is view-dependent based on what is in the scene (related to performance, etc.) or if it is due to some numerical error with the coordinate system. If starting Home changes things, then maybe it's a bug in the runtime. Does installing/removing the SteamVR Beta help?

[LoSealL]

@HelenXR Sorry for my pool English :(. I mean, your test methods change the frame, whose rendering time may also be different. Therefore you can't say that the jitter has no relation with reprojection. So I strongly recommend you to log down every frame call time, you may post here for us to analyze together :)

[HelenXR]

@echuber2 Does installing/removing the SteamVR Beta help? ---I do reinstall steamVR,the jitter remain.

@LoSealL ok, but "log down every frame call time " is refer to "VRFrames.csv" or the time when IServerTrackedDeviceProvider::RunFrame() interface be called?

[LoSealL]

@HelenXR RunFrame() is not stable, do the former.

[HelenXR]

OK,upload two Frame files in here

[LoSealL]

@HelenXR It's obvious now, you can draw NewPosesReadyMs and WaitGetPosesCalledMs in Origin or Excel yourself. Please note that NewPosesReady is negative, because it is the time that the Pose been updated before the next frame begin. And the WaitGetPose is a function called in APP, it's blocked until 100ms timeout or the new pose been updated.

You can see that in your jitter scene, the WaitGetPosesCalledMs has many peeks, many of which are even beyond 60ms. I guess that the rendering work here is too heavy (which is usual), or the app has some bugs at this scene, that causes delay on updating poses.

In Steam VR, your rendering thread can be slow or blocked, but the final view is OK because of ATW (AKA reprojection), but it also demands that pose should be updated upto 90 Hz, otherwise the ATW may use an old pose to calibrate, which finally leads to judder

[echuber2]

I meant to try removing or installing the beta version of SteamVR, depending on whether you're already using it or not. The beta may have fixed or added a bug. It does sound like the performance problem is related to the judder, which indicates bad prediction. But if you're able to alleviate the symptoms by running Home or doing a "recenter" (not sure what this means exactly), then it sounds like there is a bug or numerical precision error related to calibration. Can you explain what you mean by recenter? If this is a normalization function of some kind, it might make sense to call it at regular intervals in your engine.

[HelenXR]

@LoSealL From the data, it's true, as you say at jitter point view,NewPosesReadMs has great fluctuation:

NewPosesReadMs of at_other_point_view relatively stable:

but you say "pose should be updated upto 90 Hz",in fact My hmd report pose in 500HZ.

about ATW,I know before the function of ATW in Oculus and OSVR platform, SteamVR is how to use/debug this function? SteamVR information is really too little, I guess you have a lot of experience in this area or document, if it is convenient to send some to me, thanks.

[LoSealL]

@HelenXR Before I draw any conclusion, could tell me whether you are developing an HMD hardware (driver and related interfaces, using ) or some vr games (using , or just Unity/Unreal engine) ? Of course, if this is not confidential.

P.S The figures above is the cause, but not the root cause. So I want to figure out why they are delayed/blocked.

[echuber2]

The framerate of graphics rendering is what needs to be 90 Hz. 500 Hz sounds like the polling rate on the IMU or something. The jittery view shows that new poses are supposedly ready before WGP gets called (at 0?) -- This looks like you have a huge performance problem that is causing the entire render sequence to stall, although the HUD reported no frames dropped. As several of us have mentioned before, you should try putting a manual framerate report to console, that is write the current measured framerate to stderr each frame. That is what should be at 90 Hz...

[HelenXR]

@LoSealL I develop in hardware driver under openvr driver api(using openvr_driver.h) ,contain HMDand other hmd 's peripheral(like six dof module nolo & ximmerse),This part of the code , I built an open source project openvr_survivor

[HelenXR]

@echuber2
yes, IMU rate is 500HZ. How can i "try putting a manual framerate report to console"I haven't had any experience in this field yet.

[echuber2]

@HelenXR Make sure all vsync is turned off for the graphics driver (open Nvidia/AMD control panel and make sure all vsync and triple buffering is forced OFF). Then, in your code, you need to measure the amount of time elapsed between frame loops. So at the end of your frame rendering loop, put a check for the amount of system time passed since the last loop. Then the FPS is 1 / (time elapsed in seconds). However, this isn't great because it calculates too often. Instead, you can wait for at least one second to pass (accumulating a buffer of elapsed time) while also counting how many frames are generated. Then dump a message displaying this count to the standard error.

[LoSealL]

@echuber2 I guess, @HelenXR can't do that because he uses IVRDisplayComponent, hence the Present or Submit are all handled in VRCompositor instead of user driver, 'cause this API doesn't handle any frames.

@HelenXR You can try a black magic:

In your PoseUpdate thread, call TrackedDevicePoseUpdated twice, something like:

while (!VRServerDriverHost()->IsExiting())  {
    DriverPose_t &pose = GetPose();
    if (pose.poseIsValid) {
        /**
         * @note: A tricky way
         * The head position will blink between two updated values,
         * so call update twice to avoid head position blinking.
         */
        VRServerDriverHost()->TrackedDevicePoseUpdated(m_unObjId, pose, sizeof(pose));
        VRServerDriverHost()->TrackedDevicePoseUpdated(m_unObjId, pose, sizeof(pose));
      }
}

[echuber2]

Sorry for my lack of understanding. Would it not still be possible to insert the FPS debugging information in the application itself, rather than in the driver? That was what I meant.

[HelenXR]

@LoSealL I try do this like you say:

    while(m_bReportPoseThreadState){
               ......
        if ( m_unObjectId != vr::k_unTrackedDeviceIndexInvalid )
        {
            vr:DriverPose_t pose = GetPose();
            vr::VRServerDriverHost()->TrackedDevicePoseUpdated( m_unObjectId, pose, sizeof( DriverPose_t ) );
            vr::VRServerDriverHost()->TrackedDevicePoseUpdated( m_unObjectId, pose, sizeof( DriverPose_t ) );

        }
        ......
    }

the shake is remain. about IVRDisplayComponent I don't know much about it ,Do you have some other information about the interface except openvr's wiki?

[LoSealL]

@HelenXR I saw your survivor code, no problem there. What's your test app?

[HelenXR]

@LoSealL a free game:"InCell VR" in steam store

[HelenXR]

In short, in part of the game to find the jitter point, in the shake, after the implementation of the reset, the jitter will not occur in 360 degrees In the jitter, see the frame lost more frequently, as shown in the chart above. LoSealL should be said. Thank you again for all helps

[HelenXR]

The final solution to this problem is to modify the HID report four elements of the format. Previously used OSVR HID protocol to convert i, j, k, w from floating-point number to int16 , In the openVR driver will convert int16 into floating-point numbers. This process loses part of the accuracy of the data, passing around the point (0, 90, -180), the gesture has a blank area, resulting in jitter. Finally, this program is eliminated by passing floating point numbers directly (without conversion) thanks again for all.