[OTE 2017 Vision] – Technical ideas and clean up of the engine

[damiel]

Biel's vision is a good foundation to build on and i would like to further enrich it with technical ideas to make the engine a more stable bases for development, easier to maintain and extend and more accesible to content creators.

The codebase is quiet big with alot of moving parts, but some of these parts are outdated or legacy stuff which makes it unnecessarily hard for our small team to maintain it, examples for this are cg shaders and their translation into glsl/hlsl, flashbased gui system, 3rd party libs source code which is just "drop in" in the codebase, complex handling of these 3rd party libs but also git submodules.

My idea would it be to firstly make the engine more "equal"on multiple platforms, one step in that direction would be to finally deprecate cg shaders in our engine and replace them with a platform-independent alternative, imho the only option we have here is glsl as shader language, this means the logic to parse and handle cg shaders should be removed by a proper glsl shader parser.

Similair the flashbased menu should be removed by an alternative, we have already cegui integration but i propose that we instead try to reenable the old doom 3 gui system as it is a battleproven and has atleast some consumers within the doom3 community like for example the darkmod aka the is knowledge available how to create guis using it.

3rd party libs is a more difficult topic, we have code just dropped in as it and we also have a certain amount of git submodules. Both have certain problems, 3rd party libraries that are just dropped in like that dont receive any updates from upstream, this includes security related fixes but also potentially useful new features which we might want to integrate. Our git submodules that we have are already a bit better in that we have a copy of the upstream source which we can update easily at any time, but they also have the problem that the library in question already needs to be available via git or that we maintain the source code ourselves in git which adds alot of maintainance burden for our small team. Because of that i suggest that we remove the dropped in versions of 3rd party libraries and git sub modules and instead refactor our cmake build configuration to be smarter and find these required libraries itself. Potential devs and packagers are then free to provide the required libraries in any suitable way.

Reducing the codebase like that while make it easier to maintain it but we also what to make our engine more attractive for content creators, one way to do that is to add more modern content formats, in terms of 3d models ahve have for example "only" stuff like md3, md5, lwo and probably something else and it looks similair on the audio side. We should aim to add newer formats to the engine like OpenGex, glTF 2.0 but also older but widely used formats like ogg to give content creators more ways to use their work with our engine.

But content creators dont only need more available formats but also tools to work with, this means we also need to bring back the old engine tools the original version of doom3 had. These tools are sadly not platform independent and need to be reworked. We already have a simple light editor as proof of concept thanks to Daniel's work.

(First draft of my proposal, will update it as i get more ideas)

[BielBdeLuna]

I like it all, I propose we merge the two proposals

[damiel]

Maybe one more addition would be to finally deprecate our usage of sdl 1.2 and switch on sdl2 by default, this will also fix OTE running on macOS as it relies on code from sdl2 to work properly.

But yes i think merging the proposals is the way to go here as my ideas are meant as enrichment anyways.

[BielBdeLuna]

and also deprecate the use of JPG for textures, it's a lossy compression for textures, and we already have TGA and PNG for this that aren't lossy.

today I'll post the combined proposal.

[kortemik]

I don't have time to commit but I'd prefer something playable, a level, a character and perhaps a weapon or two and few targets to shoot.

Otherwise it's just fun programming practice.

On Sat, Sep 23, 2017 at 12:01 PM, Biel Bestué de Luna < notifications@github.com> wrote:

and also deprecate the use of JPG for textures, it's a lossy compression for textures, and we already have TGA and PNG for this that aren't lossy.

today I'll post the combined proposal.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/OpenTechEngine/Discussions/issues/49#issuecomment-331621313, or mute the thread https://github.com/notifications/unsubscribe-auth/AA19Oj6BJ59V5ywRDNH2mPiTuz_SGBDbks5slMjVgaJpZM4Pf9_3 .

[BielBdeLuna]

Seeing that the changes @damiel proposed are of a general nature, and are more argumented. I might merge the two proposals as one but I won't mix them, so the length of his argumentation don't conflict with the shorter nature of my proposal for a stepped process.

[BielBdeLuna]

Open Tech Engine is creative cooperation applied to game development. Open Tech Engine Achieves cooperation through creative means. Open Tech Engine strive for meaningful functionality as the basis for different applications.

A stepped process:

1 - building a meaningful community. The best way to hold a community together is trough having a modifiable platform, and having a genuine interaction with the community

1.1 - build a modifiable platform to have a modifiable platform means to have a set of assets that people can use as basis for their mods/games/applications the best way to have a first modifiable platform is to create those assets in an organized way

1.1.1 - building the first assets in an organized way the easiest way to create those assets would be to create meaningful assets, and identify what assets to build would be far easier as a part of a coherent game, since the engine is the byproduct of an existing game, the easier game to create would be something similar to that existing game

1.1.1.1 - building a first game since we don’t have a big team, the best bet would be to create a “vertical slash” a couple of levels, or a single level, two to three weapons, and two to three enemies, with that “vertical slash” in place levels could already be created and mods could already be done

1.1.1.1.1 - building a vertical slash we could copy existing mods or games, beside the game the engine is a byproduct of, as our “vertical slash” here is a list of proposals of single or couple of levels long mods that could be interesting as basis for our initial “vertical slash”:

doom - The City of The Damned : Apocalypse - video - link quake - Plumbers Don't Wear Ties (But They Do Carry Shotguns) - link quake - The Altar of Storms - video - link quake 3 - The edge of forever - video - link quake - The Marcher fortress - video - link quake - Firetop Mountain - video quake - The Horde of Zendar - video quake - Forgotten Sepulchre - video - all last three within this mod

we could have something build like one of them updated with the current improvements the engine has, that would serve to gather the limits of the engine we have on our hands, and a trial to resolve bugs from the new improvements, it could be something meaningful enough to stand on it’s ground but not overly complex to build.

1.1.2 - building the community pro-actively since now we have a small asset pool, we could try to build a community out of mapping, that would serve in more than one way, it would build community out of the creations of the community. And would serve to create new assets and empower tool creation. Sort of: http://www.thedarkmod.com/missions/

1.1.2.1 - manage the community with creativity we could have a lively community with mapjams, or even with yearly competitions, sort of: https://www.quaddicted.com/

1.1.2.2 – guest-mapping proposals for known mappers to guest as mappers for our engine as propaganda of the community, let those that inspired us also inspire others on our behalf.

1.1.2.3 – perpetual campaign for the engine and it’s possibilities every time we get a new feature create screen shots and didactic videos that attract new talent to the community and pro-actively distribute the videos on the appropriate social networks, sort of: http://www.celephais.net/board/news.php

1.1.3 - enhancing the community possibilities we could also build community from a asset point of view, with a repository of assets so everyone can supply their created assets so all in the community benefit from the community efforts, like this: http://www.realm667.com/index.php/en/

1.1.4 - enhancing the reach of the engine building other «vertical slashes» for games that differ from the original game, the engine is byproduct of, would be wiser at this stage so the possible applications of the engine are enhanced

2 - enhancing the technology of the engine building better technology should be an end goal that would be beneficial to all anytime

2.1 – recognizing the outsider work works outside the community are also valuable, let’s people into the community but doesn’t require it, so a proactive attitude towards outsider work should be the normal attitude

In order to further enrich the stepped process with technical ideas to make the engine a more stable base for development, easier to maintain and extend and more accesible to content creators.

The codebase is quiet big with alot of moving parts, but some of these parts are outdated or legacy stuff which makes it unnecessarily hard for our small team to maintain it, examples for this are cg shaders and their translation into glsl/hlsl, flashbased gui system, 3rd party libs source code which is just "drop in" in the codebase, complex handling of these 3rd party libs but also git submodules.

My idea would it be to firstly make the engine more "equal"on multiple platforms, one step in that direction would be to finally deprecate cg shaders in our engine and replace them with a platform-independent alternative, imho the only option we have here is glsl as shader language, this means the logic to parse and handle cg shaders should be removed by a proper glsl shader parser.

Similair the flashbased menu should be removed by an alternative, we have already cegui integration but i propose that we instead try to reenable the old doom 3 gui system as it is a battleproven and has atleast some consumers within the doom3 community like for example the darkmod aka the is knowledge available how to create guis using it.

3rd party libs is a more difficult topic, we have code just dropped in as it and we also have a certain amount of git submodules. Both have certain problems, 3rd party libraries that are just dropped in like that dont receive any updates from upstream, this includes security related fixes but also potentially useful new features which we might want to integrate. Our git submodules that we have are already a bit better in that we have a copy of the upstream source which we can update easily at any time, but they also have the problem that the library in question already needs to be available via git or that we maintain the source code ourselves in git which adds alot of maintainance burden for our small team. Because of that i suggest that we remove the dropped in versions of 3rd party libraries and git sub modules and instead refactor our cmake build configuration to be smarter and find these required libraries itself. Potential devs and packagers are then free to provide the required libraries in any suitable way.

Reducing the codebase like that while make it easier to maintain it but we also what to make our engine more attractive for content creators, one way to do that is to add more modern content formats, in terms of 3d models ahve have for example "only" stuff like md3, md5, lwo and probably something else and it looks similair on the audio side. We should aim to add newer formats to the engine like OpenGex, glTF 2.0 but also older but widely used formats like ogg to give content creators more ways to use their work with our engine.

But content creators dont only need more available formats but also tools to work with, this means we also need to bring back the old engine tools the original version of doom3 had. These tools are sadly not platform independent and need to be reworked. We already have a simple light editor as proof of concept thanks to Daniel's work.

[BielBdeLuna]

I would like to add also the following provision for changing such vision and stepped process:

Simple provisions for a chance for change of relevancy

In order to not block change, refinement and improvement, and in order to stay flexible enough as well as relevant. The re-evaluation of this vision as well as the stepped process is encouraged ( maybe every year? )

Change of the Vision of the project as well as the stepped process happens if there is a win in the proposed vote by simple majority ( half the votes reached plus another vote ), and a minimum of participation of a fourth of the whole team behind OTE ( in case of a unclear a forth of the team, it’s the last participant after clearing the fourth of all the team ) this should suffice or count as first round.

In case of not reaching a minimum of participation, and only in this case, once cleared that first round, if the interested party still want to pursue a vote for change, there is a chance of a second round with simple majority of the vote and no minimum of participation, so change is possible and not blockable by abandonment of the project by a significant portion of the team, so the project lives on with a new vision and a possible future.

and also a technical proposal of mine:

Deprecate the use of JPG for textures, it's from the Quake3 era, and no longer used in Doom3, it's a lossy compression for textures, and we already have TGA and PNG for this that aren't lossy.

what do you people think?

[BielBdeLuna]

this has my vote

[kortemik]

+1

[damiel]

+1

[BielBdeLuna]

vote passed! :+1: