Question about ej01 texture pack

illwieckz commented 7 years ago

I have done one texture repository per texture package. By the way, the ej01 texture package by @EmperorJack is currently split into three parts:

6,9M    tex-ej01-clean_src.pk3dir/.git
4,4M    tex-ej01-common_src.pk3dir/.git
8,2M    tex-ej01-ice_src.pk3dir/.git
20M total

common contains normal maps for clean and ice texture sets
clean is the default ej01 texture set
ice is an icy variation

It looks too fine-grained to me to have three repositories for them, also, adding a texture in one of them will require to add the normal map in the repository, multiplying commits across repositories and this kind of submodule dance will be painful. Also, the filesystem layout already splits them:

textures/shared_ej01-clean_src
textures/shared_ej01-common_src
textures/shared_ej01-ice_src

scripts/shared_ej01-clean.shader
shared_ej01-ice.shader

So there a very clean separation is still kept if we merge the three repositories.

@EmperorJack, what do you think about it? I ping @Viech if you has something to say about it.

Viech commented 7 years ago

I figure the idea was that a map could opt to only load the clean or the icy version, to speed up map load time. However, it should not make a noticable difference as the engine would not load a texture that is not referenced within a used shader into memory anyway. Hence, merging them should not have any undesired circumstances, unless I miss another reason why @EmperorJack packaged them like that.

EmperorJack commented 7 years ago

Hey guys. I can't remember the exact reason I split them up but I believe it was either to speed up the map load time, reduce the size of dependencies needed if the map only needed one of the texture sets or purely for ease of separating the sets in the map editor.

I think you can have shaders appear in different directories in the editor purely by changing the path in the shader file. If this is the case merging the packs should be fine. Would you like me to package the latest version for you guys this way?

Also how are you guys intending on using these asset repositories in your development / deployment pipeline? Is there some documentation I can read?

illwieckz commented 7 years ago

Would you like me to package the latest version for you guys this way?

@EmperorJack My current goal is to get currently used assets properly tracked on a repo. We are not enforcing artists to do all things the git way if they are afraid of this or just don't care: in most case maintainers would merge their stuff the same way we already do with pk3s… but people who knows how to use Git like you, @JacksonTech and @IngarKCT (who are already tracking maps or even texture set on GitHub) can contribute the git way. The best way would be to create a prerelease branch that will be rewritten and compacted just before merging to master at release time to ensure the git repository will not grow so much.

Also how are you guys intending on using these asset repositories in your development / deployment pipeline? Is there some documentation I can read?

Well, I'm just waiting to publish my WiP stuff because it would save me a lot of text to write to explains how it works. There is some stuff it's quicker to show than to explain. I have wrote some wall of texts in some places but you will lost your time reading me trying to explain what I want to show instead of just reading me explaining what I have to show. Let's try to summarize previous walls of text.

The needs for asset repositories are:

taking care of originals for assets we are using
having a change history with enough metadata like contributor/date/comment
being able to tag versions
being able to pick a revision if needed
being able to reproduce the asset build
allowing people to build against originals
being able to build prerelease stuff, even assets

Having a revision history also means a smart tool can build a package shipping only modified files between two commits (for example: since last tag).

About « how to use them », it's simple:

# get original assets:
git clone --recursive https:/…/UnvanquishedAssets
# prepare dpkdirs: preview and shader generation etc.
cd …/UnvanquishedAssets ; make build
# build maps against original lossless textures
# no uncommon format or issues related to uncommon format:
q3map2 -fs_pakpath …/UnvanquishedAssets/src
# build test dpkdirs: compile maps, play the crunch dance etc.
cd …/UnvanquishedAssets ; make build
# test built things (in dpkdirs):
daemon -pakpath …/UnvanquishedAssets/build/test
# build release dpks:
cd …/UnvanquishedAssets; make package
# test release dpks:
daemon -pakpath …/UnvanquishedAssets/build/pkg

It comes with a build tool that recognizes all the files in a source tree (no need to write Makefile lines per file) and allows mappers to define per map build options. q3map2 was heavily modified to leverage all these build facilities (q3map2 is now a compiler that understands some semantics like what is a source file, what is an out file and what is a library to build against (pakpath), enabling out of tree compilation. Well, in fact there is no need to call q3map2 by hand: just write map configuration file, set PAKPATH env var, then type urcheon build.

illwieckz commented 7 years ago

These are example of Makefiles, there will be no need to write them, I'm adding a bootstrap command to the tool to write them (and they will be there just for convenience).

[Edit: the repository names depicted in those examples are WiP stuff, and some repositories are expected to be merged]

illwieckz commented 7 years ago

So, the 3 packages were merged, the repository for the ej01 texture set is now hosted there: https://github.com/UnvanquishedAssets/tex-ej01_src.dpkdir

illwieckz commented 7 years ago

@EmperorJack see this forum post: https://forums.unvanquished.net/viewtopic.php?f=36&t=2108&start=10#p17826 :wink:

UnvanquishedAssets / UnvanquishedAssets

Question about ej01 texture pack #1