search

Toni-SM / semu.xr.openxr

OpenXR compact binding for creating extended reality applications on NVIDIA Omniverse
MIT License
31 stars 6 forks source link

I have some questions about HMD resolutions & stereo recfitication #8

Open PARKBONG opened 1 year ago

PARKBONG commented 1 year ago

Hello, I wonder some problems are realated with my local device or not . I would appreciate it if you could answer my question.

  1. stereo rectification What is the best way to determine optimal stereo rectification? Even if I set the empirically optimal rectification value for the center of the current lens, the left&right eye focus is mismatched on side of the lens. Is it same in your case? or does it just my local system & hardware problem?

  2. The resolution on my setup seems to be little lower than steamVR Home(on Ubuntu) or Omniverse XR(on Windows). Are you same?

  3. Is there any parameters that is specialized for your local device (i.e., vive pro 1) on your code? I wonder if your source code is perfectly compatible with my device (i.e., vive pro 2)... or can I solve these problems by using vive pro 1.

thanks!

PARKBONG commented 1 year ago

For example, should I change this to use my vive pro 2 without any problems?

Toni-SM commented 1 year ago

Hi @PARKBONG

  1. I do stereo rectification manually by modifying the values while using the HMD... Current rectification only involves rotation around the axis. The manufacturing rotation and distance between the lens are obtained from HDM using the OpenXR API (as implemented here). But for some reason, and according to my tests, only the rotation needs fine-tuning (rectification). If necessary, translation information could be added to the rectification...

  2. The current implementation uses the old API of the omniverse kit graphical window, image capture, and preprocessing (flip and fit). This is done in Python and CPU (NumPy arrays). Therefore, using high-resolution images (the default setting of the HMD) slows down the simulation and rendering in the HMD. Hence, it is recommended to lower the resolution of the HMD in the SteamVR configuration, for example (as indicated in this other post: https://github.com/Toni-SM/semu.xr.openxr/issues/4#issuecomment-1242213227)

    Ideally, the images rendered by the simulator should be accessed directly on the GPU. I have not had time to test if this is possible with the new viewport API...

  3. No. The OpenXR API is designed to work equally for all supported devices.

    However, it is true that interaction profiles do exist (more about it here: https://roderickkennedy.com/dbgdiary/openxr-interaction-profiles). This allows access to input and output sources of the controllers and other buttons (to increase or decrease the volume from the HMD, for example) for each manufacturing... but it is only related to the input/output sources.

    The rendering API is standard for all supported devices. The only thing that can change is the support of different rendering libraries like Vulkan, OpenGL, or DirectX, for example...

PARKBONG commented 1 year ago

thank you for your detailed reply. I have an additional question.

  1. After detailed stereo rectification, does your left/right eye match perfectly? In my case it aligns almost, but I feel a little uncomfortable on the side of the lens.
Toni-SM commented 1 year ago

Sorry for the late reply.

Mmmm, maybe an extra rectification can be performed by adjusting the camera's horizontal and vertical offsets from the Python code... I would like to have some time to try it :sweat_smile: :see_no_evil:

Screenshot from 2022-12-01 11-22-07

PARKBONG commented 1 year ago

Thank you for your reply. One thing I'm suspicious about now is image distortion (e.g. pincushion distortion rate of cv2 images) rather than camera offset...