veeenu / hudhook

A videogame overlay framework written in Rust, supporting DirectX and OpenGL
MIT License
207 stars 30 forks source link

A way to work together with detour ? #154

Closed geekerlw closed 9 months ago

geekerlw commented 9 months ago

Hello, very happy to found this project. I tried to make a rust dll mod to work with other c++ mods for richard burns rally game, but meet some errors:

Forgive me for having so many questions, emmm.

veeenu commented 9 months ago

Hi! Thank you, glad the project is useful to you!

the game already has many mods, they are hook the game's dx9 engine by detour or another asm inject, so when I tried to add this hubhook, other mods raise "detour failed createdevice", I guess the minihook is hooked by dllmain, other mods is hooked later by detour?

Interop with other mods has never been an explicit goal of the library, so I am not sure how to address this. My assumption is that once a function is detoured in any sort of way you can't really chain it with something else but I may be wrong. If you have strategies to suggest to solve this, I'm up for hearing them!

I'm not sure if this kind of support is something that I could ever offer, my hunch is that this is something that every detour strategy does slightly differently and usually every game modding community has their own mod loader with their own logic and supporting them all is a pipe dream. If someone is willing to do the research to figure out all the possible corner cases in an exhaustive way, I'd be glad to offer support in hudhook, but for me personally it's not a goal at the moment. As usual, PRs are always well accepted.

another question is I already get the d3d device and window handle with my c++ layer, and now I provide it to rust by ffi, so can I use the origin c++ d3ddevice pointer to initialize the hudbook ?

It shouldn't be necessary with the latest version of the library, which is going to hit 0.6 release soon. A Direct3D 12 device is created in the rendering engine and it is separate from the game's render loop anyway.

And advanced usage, I already provide a hooked callback to rust, can I direct call render function in the callback ?

Not at the moment, no. hudhook is meant to be an end-to-end framework that does everything from hooking to rendering. If you are already hooked into rendering functions and are using C++, you'd be probably better off just using dear imgui's first party backends directly since you don't really need much more code than that and you save yourself the hassle of Rust FFI.

last question, the game support vr, the vr mod is hooking the dx9 also, so I want to render imgui frame direct by the origin dx9, so I can see the imgui texture in the vr world. I see you create an alone dx12 to impl renderer engine, or maybe you can provice the rendered texture out, so we can get the texture to call openvr_api and settexture to openvr ?

This is not possible at the moment but it's something that could be feasible eventually. The new renderer paradigm is a render pass to an offscreen surface and a composition step on top of the game's hwnd. This last step could be made flexible and could simply return a texture which you could use in openvr. At the moment things are kinda murky and I haven't settled on a solution for this, but this will probably be necessary to solve #151 anyway.

Forgive me for having so many questions, emmm.

No worries at all! Always glad to help 😄

geekerlw commented 9 months ago

After read source codes, maybe I can implement a custom Hook just like ImguiDx9Hooks, in my Hook only need to hook another address and provide the game's HWND.

geekerlw commented 9 months ago

Tested, rewrite a ImguiDx9Hooks with a given d3ddevice and direct access the vtable()->Present, it works. So the only thing is add a feature to support return a texture to support use in vr, once it done, I'll create a PR, close this issue now, thanks.

veeenu commented 9 months ago

Thank you, I'll gladly check it out!