coop-deluxe / sm64coopdx

An official continuation of https://github.com/djoslin0/sm64ex-coop on sm64coopdx for the enhancements and progress it already has.
https://sm64coopdx.com
322 stars 56 forks source link

Adding Raytracing/RTX support to sm64coopdx by using the RT64 RenderingAPI #277

Closed iZePlayzYT closed 3 weeks ago

iZePlayzYT commented 1 month ago

[Integration of RT64 Support into sm64coopdx]

Background and Context: This pull request focuses on integrating RT64, a raytracing rendering API, into the sm64coopdx project. The main branch of sm64coopdx does not currently support RT64, so this integration required significant modifications and careful merging of changes from specific branches and repositories.

To implement RT64, I created a branch named rt64_dynos within my sm64coopdx repository (https://github.com/iZePlayzYT/sm64coopdx). The main branch of my repository, which is also used in this PR, is identical in content and state to the rt64_dynos branch. Initially, a few days ago, I attempted to port the RT64-related code from the fsr branch of the sm64rt repo (https://github.com/DarioSamo/sm64rt). However, I decided to abandon that approach due to incompatibilities and instead started from scratch using the tester_rt64alpha branch of the Render96ex repo (https://github.com/Render96/Render96ex), which, like sm64coopdx, supports DynOS.

After extensive work, I successfully merged the RT64 changes from the tester_rt64alpha branch into rt64_dynos. This effort allowed the project to compile without errors. However, I encountered significant graphical issues during runtime.

Graphical Issue and Debugging: When launching the game, a graphical error occurs, leading to an incomplete render where only the ImGui F1 menu is responsive. The rest of the game freezes, which indicates that the issue is related to the graphics rendering process. Through debugging, I discovered that two specific shader IDs, both starting with 0x490EB (as seen in the attached screenshot), are repeatedly attempted to be loaded by the function gfx_rt64_render_thread_preload_shader. These attempts happen continuously after the last console output in the game, suggesting that these shaders are either missing or incorrectly referenced. Despite the successful compilation, the rendering process fails due to these issues, resulting in the graphical errors seen.

Challenges and Unresolved Issues: In addition to the shader loading issue, there were two critical aspects that I couldn't successfully merge into my rt64_dynos branch from the tester_rt64alpha branch of Render96ex:

These issues are specifically related to the recent work on the rt64_dynos branch, which is derived from the Render96ex tester_rt64alpha branch. The rt64_normal branch, which was previously based on the sm64rt fsr branch, is no longer relevant, as that approach was cancelled in favor of starting fresh with rt64_dynos.

Request for Assistance: I am seeking help to resolve the graphical rendering issues, specifically related to the repeated attempts to load shaders with IDs starting with 0x490EB. Any guidance on properly implementing or bypassing these shaders would be highly appreciated. Additionally, advice on how to resolve the basepack and load_texture issues within the code would also be beneficial.

I’ve also created a video that shows the compilation and startup process of this integration. The video demonstrates the successful compilation, the startup, and the current graphical issues. Watching it might provide more context and help with troubleshooting. You can watch it here: https://youtu.be/VempTQbSfQ0.

Attachments:

Thank you in advance for your assistance and collaboration in resolving these issues to fully integrate RT64 into sm64coopdx!

image image

iZePlayzYT commented 1 month ago

I've just created the changes branch to assist with debugging, particularly in comparing the non-RTX and RTX versions. This branch includes two commits:

By focusing only on the affected files in these commits, it should be easier to pinpoint the exact differences between the non-RTX and RTX versions, helping with the debugging process.

JerrySM64 commented 3 weeks ago

There is one major blocker: Players using anything other than Windows cannot use the RT64 renderer as seen here:

image

iZePlayzYT commented 3 weeks ago

@JerrySM64 Try removing this macro (see screenshots) in the file src/pc/gfx/gfx_rt64.cpp, after that try recompiling with: "make clean && make RENDER_API=RT64" image image