Would VR be possible?

[raleesound]

So I know that this is almost certainly out of scope for what you are trying to do here, but would it be possible to add VR support into what you have done here? I would be interested in taking a crack at it, but I mostly wanted to know if any of this is theoretically possible with what you have created here (which is awesome, BTW) before I start spending a lot of cycles on it.

• Framerate -- I know there is the issue with JK barfing if the FPS gets over 47. Do you think that there is any possible way to work around this? Like forking off the script process, even into a separate instance of the game, and then grabbing the necessary information to render in a separate process? (Just one crazy idea that popped into my head about it).
• Stereo rendering -- When you are doing your injection, do we get enough data to do full stereoscopic rendering? I would imagine that we must, with how deep the rendering improvements go, but just wanted to ask.
• 6 DoF camera movement + independent hands -- Do you think that it is possible to separate the camera position from the shooting position like this? Even if we just had to hide the original weapon/hands completely and draw our own in a back channel way
• Lightsaber collision -- How difficult would it be to isolate and modify the logic around where the damage from a lightsaber strike gets calculated? Obviously, if the game is in VR, I would need to allow the player to swing the lightsaber around, but would then need custom logic for what counts as a "hit".
• Move the hud around -- How much control do we have over the placement of the HUD? And how much control would we have over adding additional HUD elements (if I felt saucy and wanted to, say, add a weapon wheel)?

Again, really just spitballing here, but Jedi Knight really would be an incredible experience in VR, with the scale and type of movement in the game. I also see that you were working on a full source port, which would naturally be a better candidate for this type of thing, but that is also a much bigger lift than what you are doing with jkgfxmod.

In any event, I have to let you know that this work you have done here is phenomenal -- this game is a huge part of my childhood, and it has become worse and worse to try and play it over the years, to the point that I had just basically given up on it until I found your amazing mod. Thank you so much for this!

[jdmclark]

Hi @raleesound

Thanks, and thanks for the suggestion.

Unfortunately, I don't think JkGfxMod would be much help in writing a VR port. JkGfxMod doesn't hook the executable, it just intercepts JK's original calls to Direct3D 5. Solving the first three problems you listed would require enough reverse-engineering of JK.exe that, at that point, I think you would be better off replacing the rendering code in situ rather than using an external wrapper like this one.

• Framerate: The strange behavior here is because JK has a hard-coded 48 Hz physics update rate and doesn't interpolate between updates. This causes stuttering. (This also affects the camera, because the camera is always attached to a thing and subject to the same update rate.) I think this is probably livable; it won't make you sick if in-game objects stutter a little as long as the camera doesn't (and you have to detach the "camera" from a thing in order to do VR, anyway).

Potential issue, JK does all software T&L. It's not a given that you can inject the HMD matrices at all; it's not a given that you can force the game to render multiple times per frame without issue; it's not a given that you can reliably pump all of those pre-transformed triangles to the GPU every frame.

• Stereo rendering: There's enough depth information in JK's pre-transformed triangles to do some things, like SSAO. You might be able to use this for a really basic (not physically accurate) parallax effect, but JK also does its own software clipping so this would look very bad near the screen edges. This is probably good enough for red-blue glasses, but it's nowhere near what you'd need for VR.

• Independent camera + hands: The player thing is both the camera and the weapon projectile source, each with different offsets (controlled through a variety of ways). Much like the overrides you'd need for the camera/renderer, you'd need to override the first person weapon mesh to use the VR controller's model matrix, and somehow hook the (iirc) fireprojectile COG verb to use the controller as the real source instead of using the source thing's position.

Altogether, this would be some fairly extensive modifications to the original game executable. It's not impossible, but the work involved is much closer to the scale of a reverse-engineering effort like OpenRCT2 than it is a wrapper like JkGfxMod.

[jdmclark]

As much as I'd like to play this, it's outside the scope of JkGfxMod. Closing.