ec-
commented
6 years ago Thank you for your work and intentions. I'm already reversed shader load order (since 2009) and commented known buggy shaders in renderer code. For future - sure, I'd like to implement performance-related ideas, piece by piece and with intermediate releases to check for bugs. Rest of the ideas is more or less discussable, I'll leave them in mind.
For some operative feedback or any suggestion - feel free to join discord channel related to this engine https://discordapp.com/invite/X3Exs4C
Chomenor
ensiform
I've been working on this project for several years now. I've been trying to propose this for ioquake3, but so far it doesn't seem to have gotten much interest, so I thought I'd try here.
Here's a general overview of the project. You can also check the readme and design doc, if you haven't seen them already.
ID Pak Precedence
In this filesystem the ID paks have precedence over all other pk3s in baseq3, compared to the original behavior where pk3s alphabetically higher than 'p' override them. Permanent mod pk3s can still be used, but need to be moved to the 'basemod' folder instead of baseq3.
This change makes the game more stable against unwanted overrides from arbitrary pk3s, including pk3s downloaded from servers. It gives the user more control because only the pk3s they specifically select by placing them in basemod (or that are part of an fs_game mod) have override privileges.
Other Precedence Changes
Certain file precedence behavior has been adjusted to better handle conflict situations and respect and the intentions of the ID pak precedence policy. These changes have little to no practical compatibility impact in normal usage, but do prevent bugs caused by badly designed pk3s. The following examples show the main precedence changes and their motivation.
Scenario: The shader "console" is defined in gfx.shader in pak0.pk3. Suppose "ql_mapconversion.pk3" defines its own "console" shader in "shadersfromquakelive.shader". Due to quirks in the standard Q3 engine, shader precedence (for uniquely named .shader files) is the opposite of normal filesystem precedence. Therefore the shaders in pak0.pk3 do not get overridden and this map conversion 'works' because q > p. However, if somebody decides to rename the map pk3 file to "mapconversion.pk3", or the user's ID pak was renamed to "zpak0.pk3", or similarly prioritized by the engine, incompatible shaders would probably be loaded and put the game in a broken state.
Solution: Shaders follow the same conventions as other assets regardless of the name of the .shader file. This ensures the ID pak, mod/basemod precedence, and similar policies work as intended instead of potentially having the opposite effect. Due to the ID pak precedence policy, maps with excess shaders such as the one described continue to work without risk of destabilizing the game.
Scenario: A map pk3 contains certain shaders/textures. It is a popular map, and other maps copy assets from this map. An amateur map author decides they want to modify one of the assets in their map, but they leave the same name, causing the change to bleed into the original map as well when their pk3 is present.
Solution: The current map pk3 has precedence over other arbitrary pk3s in baseq3. This prevents a lot of conflict headaches for map authors, and even allows some 'mod' maps that modify assets without changing the map bsp to functionally coexist with the original map (e.g. padshop/adultshop). This behavior applies only to regular maps in baseq3, not maps from an ID pak or mod pk3.
Scenario: The shader "textures/base_trim/basemetalsupport" is not explicitly defined in a shader file, but it can be used by maps to reference the jpg image file of that name in pak0.pk3. Suppose a map author decides to put an explicit shader for this image in their pk3 to do some special effect. In the standard filesystem, the explicit shader will always override the image and the change will bleed into all maps.
Solution: Shaders continue to override images in most cases, because there are valid use cases for this. However, images in the ID paks or current map are special cases and can be protected from being overridden by external shaders.
Scenario: If a map pk3 includes "textures/base_trim/basemetalsupport" as a tga file, it would override the version in pak0.pk3 in spite of pk3 name or the ID pak policy because the renderer traditionally searches for tgas ahead of jpgs.
Solution: Precedence is applied regardless of image type, so a jpg in an ID pak or mod can override a tga in baseq3. This ensures filesystem precedence policies work as intended when multiple image types are involved. Similar behavior applies to sound file types as well.
Performance Improvements
This filesystem is designed from the ground up for high performance. There are several load time optimizations, including caching pk3 index data (matched by size and timestamp) to a file for faster startup, and avoiding the need to reindex the filesystem between maps for faster map load times.
Here are some benchmarks comparing my ioquake3 filesystem port versus current quake3e with different numbers of map pk3s installed in baseq3.
Test methodology:
Results:
Notes:
Other features
Functional features:
Implementation features:
Conclusion
I'm proposing this filesystem as a possible candidate for Quake3e because it seems to fit with existing Quake3e design priorities; i.e. an emphasis on practical Quake 3 multiplayer, map support, and performance improvements.
If you would like, I'm willing to assist in porting my filesystem to Quake3e, including helping with support and technical issues and perhaps making a prototype integration. Based on a brief review of the filesystem code in Quake3e I think the port should be straightforward.