Renderer rewrite - Githubissues

xycaleth commented 11 years ago

I propose that the renderer is rewritten or at least updated to take advantage of the newer graphics techniques which have been available for the past 10 years or so (they were introduced around the same time that JKA was released).

There are many ways the current renderer can be improved. The bottlenecks in the current renderer (from an educated guess) are:

All vertex processing is performed on the CPU. All games nowadays offload this work to the GPU which is designed to handle these types of calculations (in parallel).
All vertex data must be transferred to the graphics card every frame. Using vertex buffers allows the vertex data to remain resident on the graphics card which are referenced by identifiers. Instead of sending all the vertex to render, the renderer can instead tell the GPU to render vertices from a specific vertex buffer instead.
Each shader stage is a separate 'render pass'. This means that a surface may have to rendered multiple times in order to generate the desired outcome. I half-wrote a Q3 shader to GLSL shader translator a few years ago which I may consider finishing to help with this. By doing this, most cases the shader stages should be able to be merged into one render pass.

I'll post a proposed renderer design some point in the near future (hopefully). I probably won't have enough time to code it until June/July so by posting the design, there's a chance someone else can start work on it.

mrwonko commented 11 years ago

I propose we also add (optional?) alpha sorting. The Quake 3 engine "best practice" is just using additive blend, which is commutative, but that's a very limiting workaround. The engine guarantees no sorting except for the sort shader command, which allows for manual sorting, but is only useful for things you can't get behind.

Does the engine do a Z Prepass yet? It should, to reduce overdraw.

xycaleth commented 11 years ago

What do you mean by alpha sorting?

mrwonko commented 11 years ago

Transparent surfaces (with an alpha channel) need to be drawn in back-to-front order because you can't use the z-buffer for them (transparent surfaces don't occlude stuff behind them).

Disabling Z-Testing without ordering leads to surfaces in the distance being drawn in front of closer ones, while enabling Z-Testing leads to the ones in the back not being drawn at all anymore.

phire commented 11 years ago

I suggest we target OpenGL 2.0 and OpenGL ES 2.0 (It's reasonably easy to make code which compiles on both)

However there are still a number of intel's crappy i965 GPUs floating around (in netbooks) that we might want to support. They max out at OpenGL 1.4 so maybe we want a fallback mode.

owenworley commented 11 years ago

The way ioq3 does it is to move the renderer into seperate .dll/.so/.dylibs for each pathway (Opengl1, opengl2). This is nice and flexible.

It's worth taking a look at ioq3's renderer, as they have implemented some great functionality including ssao, parallax occlusion mapping, hdr, etc. All of which is backwards compatible with base. E.g. When a model or other asset is loaded, the engine checks for the existence of an extra shader file which specifies the additional textures etc needed for advanced rendering techniques.

xycaleth commented 11 years ago

SSAO, parallax occlusion mapping and HDR are fairly easy to implement and would come in later. The main objective here is to speed up what already exists, and if done right, will allow the aforementioned techniques to be added really easily.

eezstreet commented 11 years ago

What Xycaleth said. Fix the renderer, worry about pretty stuff later.

Sent from my Windows Phone

From: Alex Lomailto:notifications@github.com Sent: ‎4/‎9/‎2013 11:25 AM To: Razish/OpenJKmailto:OpenJK@noreply.github.com Subject: Re: [OpenJK] Renderer rewrite (#54)

SSAO, parallax occlusion mapping and HDR are fairly easy to implement and would come in later. The main objective here is to speed up what already exists, and if done right, will allow the aforementioned techniques to be added really easily.

Reply to this email directly or view it on GitHub: https://github.com/Razish/OpenJK/issues/54#issuecomment-16119521

owenworley commented 11 years ago

I wasn't suggesting looking at 'pretty' stuff right away, i was pointing out that a modular system would work well here. Once the framework is created (small amount of abstraction) it allows for phire's comments about having seperate OpenGL pathways to support older systems, while also allowing encapsulated development.

eezstreet commented 11 years ago

Fair enough. As far as that abstraction is concerned, Raz0r is working on getting that set up for OpenJK MP. I believe its static linked, with Ghoul 2 part of the renderer. Either way, once he's done doing that, feel free to poke it for OpenGL 2.0 ES support. I'll have to get it ported to SP as well.

Sent from my Windows Phone

From: owenworleymailto:notifications@github.com Sent: ‎4/‎9/‎2013 12:20 PM To: Razish/OpenJKmailto:OpenJK@noreply.github.com Cc: eezstreetmailto:eezstreet@live.com Subject: Re: [OpenJK] Renderer rewrite (#54)

I wasn't suggesting looking at 'pretty' stuff right away, i was pointing out that a modular system would work well here. Once the framework is created (small amount of abstraction) it allows for phire's comments about having seperate OpenGL pathways to support older systems, while also allowing encapsulated development.

Reply to this email directly or view it on GitHub: https://github.com/Razish/OpenJK/issues/54#issuecomment-16123331

Razish commented 11 years ago

@owenworley I am currently working on adding the modular renderer, which means Rend2 from ioq3 may indeed be ported, but more importantly new renderers can be made without overwriting anyone else's work. When a renderer is ready to be used in the main branch, all it takes is a pull request.

Razish commented 11 years ago

Update: Modular renderer was added a few days ago. See the README.md for reserved renderer names =]

mrwonko commented 11 years ago

Before starting to add normal mapping and what not, please first create a draft of how it would be used by content creators (i.e. would there be a new shader (as in material) command?), since that needs to be something reasonable. There's a lot of different ways of doing this and careful consideration is advisable.

The reason for this request - besides planning always being good - is that I don't want to end up with competing standards in my upcoming fork. There should be one - the - way for content creators to go.

eezstreet commented 11 years ago

It’s important to note that there are some unused shader keywords in SP, such as bumpMap. Need to keep things consistent between SP and MP in that regard.

Sent from Windows Mail

From: Willi Schinmeyer Sent: ‎May‎ ‎6‎, ‎2013 ‎1‎:‎02‎ ‎PM To: Razish/OpenJK CC: eezstreet Subject: Re: [OpenJK] Renderer rewrite (#54)

Before starting to add normal mapping and what not, please first create a draft of how it would be used by content creators (i.e. would there be a new shader (as in material) command?), since that needs to be something reasonable. There's a lot of different ways of doing this and careful consideration is advisable.

The reason for this request - besides planning always being good - is that I don't want to end up with competing standards in my upcoming fork. There should be one - the - way for content creators to go.

— Reply to this email directly or view it on GitHub.

DT85 commented 11 years ago

It would be more accurate and less confusing to have 'normalmap' rather than 'bumpmap' as they are very different from one another.

Have a look here at Doom 3's materials: http://www.iddevnet.com/doom3/materials.php

eezstreet commented 11 years ago

Why would you refer to a bumpmap as a normalmap if it's a bumpmap?

Sent from my Windows Phone

From: DT85mailto:notifications@github.com Sent: ‎5/‎7/‎2013 8:54 AM To: Razish/OpenJKmailto:OpenJK@noreply.github.com Cc: eezstreetmailto:eezstreet@live.com Subject: Re: [OpenJK] Renderer rewrite (#54)

It would be more accurate and less confusing to have 'normalmap' rather than 'bumpmap' as they are very different from one another.

Have a look here at Doom 3's materials: http://www.iddevnet.com/doom3/materials.php

Reply to this email directly or view it on GitHub: https://github.com/Razish/OpenJK/issues/54#issuecomment-17540159

DT85 commented 11 years ago

What is said to be implemented is a normal map, not bump map.

http://en.wikipedia.org/wiki/Normal_mapping http://en.wikipedia.org/wiki/Bump_mapping

Why would you refer to a normalmap as a bumpmap if it's a normalmap?

xycaleth commented 11 years ago

Technically, a normal map is a kind of bump map :P (same with a parallax map being a kind of bump map). But I do think it would be better to use the normalmap keyword, if shaders are going to be built like they are now. Most people refer to height maps when they say bump map, so we could leave the bumpmap keyword free for heightmaps.

Razish commented 11 years ago

I like those D3 shaders, very similar to q3's format.

DT85 commented 11 years ago

True true, though they are more commonly known seperately and we shouldn't add any potential confusion for artists. But this is just an added bonus lol, I'm sure most artists won't care what it's named so long as it's written somewhere in a readme what it is. :P

Now something I'd like to see (not sure if it's possible in games) is displacement mapping. This would really bring meshes with a flat surface (walls, floors, etc) alive with depth alot moreso than normal/bump maps could ever do.

StrikerMan780 commented 11 years ago

"Now something I'd like to see (not sure if it's possible in games) is displacement mapping. This would really bring meshes with a flat surface (walls, floors, etc) alive with depth alot moreso than normal/bump maps could ever do."

The DarkPlaces Engine (Modded Quake 1 engine with a bit of Q3 code iirc) does this. Take a look at the game Xonotic for lots of great examples. (Open-source FPS based on DarkPlaces.)

ensiform commented 10 years ago

We're going to have our take on JA flavored rend2 from ioquake3 but not a full rewrite in the main repository. Closing.

JACoders / OpenJK

Renderer rewrite #54