Support Meta Quest 3 native resolution

[nowak-pl]

Meta Quest 3 has LCD panels with per eye resolution of 2064 x 2208. Setting this resolution is currently impossible as it gets rounded down to 2048 x 2208. This pull request relaxes resolution alignment from 32 to 16 pixels.

[zmerp]

The Nvidia encoder does not support resolutions that are not a multiple of 32, with the exception of a few standard resolutions like HD, FHD, QHD etc. if we request a resolution which is not a multiple of 32 the image will be stretched to fill the next up multiple of 32, which will result in a blurry image. The real fix would be to request the next up multiple of 32 resolution but render at the original requested resolution. to do this we would need support in the client and server renderer.

[nowak-pl]

We are talking about per-eye resolution, encoding is done on width * 2 so it is divisible by 32. Maybe align to 16 only for width with height keep at 32. With the changes and nvidia gpu 2064 x 2208 works for me nicely.

[zmerp]

True, for width we can align to 16 per eye. I personally don't believe this has any practical benefit, as I believe that "native screen resolution" as magical numbers don't make sense. The reason is the image is distorted because of lens correction, which even causes inconsistent pixel densities across the eye view, and only a portion of the screen is often used to render the frame.

[The-personified-devil]

The Nvidia encoder does not support resolutions that are not a multiple of 32

huh, h264 macroblocks are 16x16

[Vixea]

CI failure is unrelated I'm guessing a cargo apk update or something, is now enforcing a specific platform version 32

[zmerp]

huh, h264 macroblocks are 16x16

@The-personified-devil HEVC macroblock size is 64x64. I think the macroblock side doesn't matter, it's a limitation of the API itself

[The-personified-devil]

@The-personified-devil HEVC macroblock size is 64x64. I think the macroblock side doesn't matter, it's a limitation of the API itself

Yes it does, 1080p and co are only supported because the encoder just adds a black bar that it then cuts out during decoding again. So if the encoder wants stuff to be aligned by macroblocks (which it should, dunno why it wouldn't, except for special resolutions) and we would have to conform to that for all codecs that we have. So with the 64 for hevc we couldn't lower the per eye to below 32.

[zmerp]

Hm. Welp, so should we increase the alignment to 64?

[The-personified-devil]

Hm. Welp, so should we increase the alignment to 64

No because as we noticed per eye gets multiplied to 2, so 32 alignment per eye would be enough, and that's what it already is. Only thing I'm not sure about is how foveated encoding handles all of that/how the encoders have been dealing with it or are we actually aligning that properly?

[zmerp]

No because as we noticed per eye gets multiplied to 2, so 32 alignment per eye would be enough, and that's what it already is.

Nope, vertical resolution is still wrong.

Only thing I'm not sure about is how foveated encoding handles all of that/how the encoders have been dealing with it or are we actually aligning that properly?

We assign a new resolution that is a multiple of 32 horizontally and vertically.

[zmerp]

@nowak-pl ping?