Feature request: Redesign of the build process (each project has it own axmol lib and own axmol build settings)

[aismann]

New build design: Each project has it own build settings for the Axmol Engine Thats means:

• Axmol Engine is more a bulk of sources (core, 3rdparty, extensions, templates, etc)
• plus the tools for creating own projects easily

All Project which comes with axmol be used for testing and used as example sources for using axmol:

• All projects which be in folder tests has it "own" axmol settings
• This projects be usefull for testing/validation of axmol PRs

Each (own) project: Full controll of the axmol settings on each project

Why this build change is usefull?

• You create your own axmol lib for each own project.
• Full control of the build of axmol for each project.

@halx99, @rh101 What are you think about this "improvement"?

The big change is (the optional settings of axmol be part of each project (not global on axmol)

option(AX_ENABLE_PHYSICS "Build Physics support" ON)
option(AX_ENABLE_MEDIA "Build media support" ON)
option(AX_ENABLE_AUDIO "Build audio support" ON)
option(AX_ENABLE_CONSOLE "Build axmol debug tool: console support" ON)

option(AX_ENABLE_3D "Build 3D support" ON)
cmake_dependent_option(AX_ENABLE_3D_PHYSICS "Build 3D Physics support" ON "AX_ENABLE_3D" OFF)
cmake_dependent_option(AX_ENABLE_NAVMESH "Build NavMesh support" ON "AX_ENABLE_3D" OFF)

option(AX_UPDATE_BUILD_VERSION "Update build version" ON)
option(AX_DISABLE_GLES2 "Whether disable GLES2 support" OFF)

# default value for axmol extensions modules to Build
# total supported extensions count: 13
# extensions dependicies: COCOSTUDIO may depend on spine & dragonBones if they are present in buildset

option(AX_ENABLE_EXT_GUI "Build extension GUI" ON)
option(AX_ENABLE_EXT_ASSETMANAGER "Build extension asset-manager" ON)
option(AX_ENABLE_EXT_SPINE "Build extension spine" ON)
option(AX_ENABLE_EXT_DRAGONBONES "Build extension DragonBones" ON)
option(AX_ENABLE_EXT_COCOSTUDIO "Build extension cocostudio" ON)
option(AX_ENABLE_EXT_FAIRYGUI "Build extension FairyGUI" ON)
option(AX_ENABLE_EXT_LIVE2D "Build extension Live2D" OFF)
option(AX_ENABLE_EXT_EFFEKSEER "Build extension Effekseer" ON)

cmake_dependent_option(AX_ENABLE_EXT_PARTICLE3D "Build extension Particle3D" ON "AX_ENABLE_3D" OFF)
cmake_dependent_option(AX_ENABLE_EXT_PHYSICS_NODE "Build extension physics-nodes" ON "AX_ENABLE_PHYSICS" OFF)
cmake_dependent_option(AX_ENABLE_EXT_IMGUI "Build extension ImGui" ON "(WINDOWS AND NOT WINRT) OR MACOSX OR LINUX OR ANDROID OR WASM" OFF)
cmake_dependent_option(AX_ENABLE_EXT_INSPECTOR "Build extension Inspector" ON "AX_ENABLE_EXT_IMGUI" OFF)
cmake_dependent_option(AX_ENABLE_EXT_SDFGEN "Build extension SDFGen" ON "AX_ENABLE_EXT_IMGUI" OFF)
option(AX_ENABLE_EXT_JSONDEFAULT "Build extension JSONDefault" ON)
option(AX_ENABLE_EXT_DRAWNODEEX "Build extension DrawNodeEx" ON):

[smilediver]

We also need a better support for this. We have multiple projects in our studio and all of them will have separate Axmol sub repos in game/libs/axmol folder. Our projects must be able to update their own version of Axmol at their own pace. Since we have people that work on multiple projects at the same time, we can't have a single Axmol install. Not to mention we can't have a single install on build machines too.

At the moment we set _AX_ROOT in project's CMakeLists.txt to point where the game's Axmol sub repo is (in our case that would be ${CMAKE_CURRENT_SOURCE_DIR}/libs/axmol. But not every part of build process supports this (like glslcc). I think best way forward would be for projects to setup _AX_ROOT and setup build system to work around that.

[rh101]

@halx99, @rh101 What are you think about this "improvement"?

Each project does already have control over the build options, and it does not affect any other project, so what exactly is the purpose of this? I must be misunderstanding something, because I'm confused as to what this is for.

[rh101]

I think best way forward would be for projects to setup _AX_ROOT and setup build system to work around that.

This I agree with. Something like the _AX_ROOT variable should be the one that is used to control which version of Axmol is used, and that includes all parts of the build process (including glslcc).

[aismann]

I have a found a workaround (for win32):

Each project does already have control over the build options, and it does not affect any other project, so what exactly is the purpose of this? I must be misunderstanding something, because I'm confused as to what this is for.

Its possible now to have a stable axmol version for working on own projects and also you can have an axmol developer version or any other different axmol stuff on the same machine at the same time. Build process can be also at the same time for each own project Switch between the different axmol versions can be quickly (if it needed) for each project too

On my build machine currently running 2 different projects with different settings all is controlled over the AX_ROOT env: The small one (39 projects):

the big one (56 projects):

[smilediver]

I'm thinking of what should be done to make Axmol move away from a single install to a more "embedded library" setup. I think a good way forward would be to:

• Remove AX_ROOT environment variable.
• Add AX_ROOT variable for CMake, that would pick the engine location.
• Ensure that all CMake steps use that AX_ROOT variable.
• Update the setup.ps1 to take these changes into account.

We could leave AX_ROOT environment variable as a fallback, but that is probably not necessary.

I'm also wondering if the path to the engine could be stored in the <project root>/axmol_root file, so that CMake and other scripts like build.bat could read the path from it... but that may not be very portable if you want to have a single install setup, since you will be forced to use relative paths.

[smilediver]

Maybe CMake and other scripts could check for the presence of <project root>/axmol_root file. If that file exists, then engine path is read from it, otherwise the path is read from AX_ROOT environment variable. This would allow for both single install and embedded library setups.

[aismann]

I'm thinking of what should be done to make Axmol move away from a single install to a more "embedded library" setup. I think a good way forward would be to:

• Remove AX_ROOT environment variable.
• Add AX_ROOT variable for CMake, that would pick the engine location.
• Ensure that all CMake steps use that AX_ROOT variable.
• Update the setup.ps1 to take these changes into account.

We could leave AX_ROOT environment variable as a fallback, but that is probably not necessary.

I'm also wondering if the path to the engine could be stored in the <project root>/axmol_root file, so that CMake and other scripts like build.bat could read the path from it... but that may not be very portable if you want to have a single install setup, since you will be forced to use relative paths.

Good points. Maybe @halx99 will add this on the milestones?

The PR #1804 is only a workaround, which is working on windows with the current axmol build system.

[rh101]

Maybe CMake and other scripts could check for the presence of <project root>/axmol_root file. If that file exists, then engine path is read from it, otherwise the path is read from AX_ROOT environment variable. This would allow for both single install and embedded library setups.

This is a reasonable suggestion.

There are a few scenarios to deal with:

1 - Running axmol new ... requires that the engine exists in the path, so it will use whatever version is currently in the system path. To use a different release of Axmol, then the only thing I can think of at the moment is to explicitly specify the path to the axmol script, and then that script should only use the tools relative to the path provided.

2 - Running axmol build may easier to handle, as you suggested, but it would mean the axmol script would need this feature, which wouldn't be available in older releases. Perhaps an easier way to handle this would be to just add an "axmol" bootstrap script in the project folder, which can reference the required axmol path, and this in turn calls the required engine axmol script explicitly. That way when you run the "axmol" command in the project folder, it would run the local axmol script, which simply then calls the true script in the specified axmol path. The advantage of this is that it may possibly be backwards compatible, since it would simply be a matter of adding the axmol script (axmol.bat, axmol.sh or axmol.ps1 etc.) to the root folder of the project.

[halx99]

I prefer to add a parameter to axmol new , i.e. -isolated copy all engine folders core, 3rdparty, extensions to new project

[rh101]

I prefer to add a parameter to axmol new , i.e. -isolated copy all engine folders core, 3rdparty, extensions to new project

That would solve the code compatibility issue, and is pretty much how cocos2d-x handled it. What it may not solve is the issue of tools and cmake scripts, which, in newer versions of axmol, may no longer be compatible with older projects. What may be required is to copy everything across to the project, excluding the things that the project will not use, such as tests/templates etc..

[halx99]

cmake & 1k folder also need copy

[halx99]

@aismann please help test it

[smilediver]

I'm against the copying approach as this would disconnect the sources from Axmol repo.

Currently we use Cocos and we integrate it by simply having Cocos repo as a submodule in our game repo in <game repo root>/libs/cocos2dx folder. Right now we're trying out Axmol with the same approach with Axmol repo added as a submodule in <game repo root>/libs/axmol, and we simply set _AX_ROOT in game's CMakeLists.txt to point to ${CMAKE_CURRENT_SOURCE_DIR}/libs/axmol. As our previous experience with Cocos shows, this setup works really well, it's very easy to update the engine when there's a need, you can see the history, you always know what is the actual commit that is being used, do custom commits for hot fixing bugs, etc.

Copying the sources would complicate the maintenance, since you lose the link to main repo. For example, right now I can fix Axmol bugs and commit straight to the Axmol submodule. Then I can simply cherry pick those fixes into a separate branch for making PR's to Axmol's upstream repo. Then later I can easily update Axmol by just switching the commit used for the submodule. With the copying sources approach this will be a lot more difficult.

[rh101]

I'm against the copying approach as this would disconnect the sources from Axmol repo.

The isolated project support is optional, and for developers that want the same default behaviour as Cocos2d-x. Correct me if I'm wrong, but from what I see in the changes made to the cmake files of both the template CMakeListst.txt and AXBuildSet.cmake, it actually enhances the support of using the Axmol repo as a submodule of your project, although it would be in the path <game project root>/axmol instead of the path you're currently using, <game repo root>/libs/axmol

[aismann]

@aismann please help test it

Of course ;)

[rh101]

So, the changes committed in https://github.com/axmolengine/axmol/commit/197f2adfcd1e23b35f4520eeeccc655acdbe3356 do make it a lot easier to work with git submodules, but with a few things to keep in mind.

The first requirement is that you would need an existing Axmol installation in order to create the new project.

For example, creating a project with: axmol new -p org.axmol.test -d . l cpp subtest

Next add axmol as a submodule in the project: git submodule add https://github.com/axmolengine/axmol axmol git submodule init git submodule update

Now comes the bit that may possibly be an issue.

If you already have an axmol installation that is in the environment path, and you use the build command, such as: axmol build -p win32 -c

It will map the tool paths (such as glslcc and nuget) to that installation, which may be fine, but also may be an issue if it's a different version than you were expecting (such as with future axmol updates). For example, the axmol installation is in G:\game_engines\axmol, so this happens:

1kiss: proj_dir=G:\dev\public\tests\subtest, external_prefix=G:\game_engines\axmol\tools\external
1kiss: Using glslcc: G:\game_engines\axmol\tools\external\glslcc\glslcc.exe, version: 1.9.4
1kiss: Using cmake: C:\Program Files\CMake\bin\cmake.exe, version: 3.29.0
1kiss: Using nuget: G:\game_engines\axmol\tools\external\nuget\nuget.exe, version: 5.5.1

A simple way to fix this would be to have simple script, such as the following, in the game project folder (for example, let's call it <game project>\axmol.ps1):

$localAxmol = (Join-Path $PSScriptRoot 'axmol/tools/console')

if (Test-Path -Path $localAxmol) {
    $scriptPath = (Join-Path $localAxmol '/axmol.ps1')
    Invoke-Expression "$scriptPath $args"
} else {
    Invoke-Expression "axmol $args"
}

You run it as: ./axmol build -p win32 -c

If it detects that there is a local axmol, it uses the local installation of axmol, which results in these paths:

1kiss: proj_dir=G:\dev\public\tests\subtest, external_prefix=G:\dev\public\tests\subtest\axmol\tools\external
1kiss: Using glslcc: G:\dev\public\tests\subtest\axmol\tools\external\glslcc\glslcc.exe, version: 1.9.4
1kiss: Using cmake: C:\Program Files\CMake\bin\cmake.exe, version: 3.29.0
1kiss: Using nuget: G:\dev\public\tests\subtest\axmol\tools\external\nuget\nuget.exe, version: 5.5.1

As you can see, they all point to the local axmol installation, not the one in the global path. This also solves the issue of the game project being copied to a new system without an existing axmol installation, since there would be no axmol command to use.

[aismann]

So, the changes committed in 197f2ad do make it a lot easier to work with git submodules, but with a few things to keep in mind.

The first requirement is that you would need an existing Axmol installation in order to create the new project.

For example, creating a project with: axmol new -p org.axmol.test -d . l cpp test

Next add axmol as a submodule in the project: git submodule add https://github.com/axmolengine/axmol axmol git submodule init git submodule update

Now comes the bit that may possibly be an issue.

If you already have an axmol installation that is in the environment path, and you use the build command, such as: axmol build -p win32 -c

It will map the tool paths (such as glslcc and nuget) to that installation, which may be fine, but also may be an issue if it's a different version than you were expecting (such as with future axmol updates). For example, the axmol installation is in G:\game_engines\axmol, so this happens:

1kiss: proj_dir=G:\dev\public\tests\subtest, external_prefix=G:\game_engines\axmol\tools\external
1kiss: Using glslcc: G:\game_engines\axmol\tools\external\glslcc\glslcc.exe, version: 1.9.4
1kiss: Using cmake: C:\Program Files\CMake\bin\cmake.exe, version: 3.29.0
1kiss: Using nuget: G:\game_engines\axmol\tools\external\nuget\nuget.exe, version: 5.5.1

A simple way to fix this would be to have simple script, such as the following, in the game project folder (for example, let's call it <game project>\axmol.ps1):

$localAxmol = (Join-Path $PSScriptRoot 'axmol/tools/console')

if (Test-Path -Path $localAxmol) {
    $scriptPath = (Join-Path $localAxmol '/axmol.ps1')
    Invoke-Expression "$scriptPath $args"
} else {
    Invoke-Expression "axmol $args"
}

You run it as: ./axmol build -p win32 -c

If it detects that there is a local axmol, it uses the local installation of axmol, which results in these paths:

1kiss: proj_dir=G:\dev\public\tests\subtest, external_prefix=G:\dev\public\tests\subtest\axmol\tools\external
1kiss: Using glslcc: G:\dev\public\tests\subtest\axmol\tools\external\glslcc\glslcc.exe, version: 1.9.4
1kiss: Using cmake: C:\Program Files\CMake\bin\cmake.exe, version: 3.29.0
1kiss: Using nuget: G:\dev\public\tests\subtest\axmol\tools\external\nuget\nuget.exe, version: 5.5.1

As you can see, they all point to the local axmol installation, not the one in the global path. This also solves the issue of the game project being copied to a new system without an existing axmol installation, since there would be no axmol command to use.

@rh101 Maybe a wiki entry for this "new" feature is also usefully?

[aismann]

@halx99 , @rh101 Can it closed? It works for me. Thanks for add this (old) cocos2dx feature again.

[aismann]

@halx99 It is possible to add this (see https://github.com/axmolengine/axmol/issues/1801#issuecomment-2049131150) on the build.ps1:

A simple way to fix this would be to have simple script, such as the following, in the game project folder (for example, let's call it <game project>\axmol.ps1):

$localAxmol = (Join-Path $PSScriptRoot 'axmol/tools/console')

if (Test-Path -Path $localAxmol) {
    $scriptPath = (Join-Path $localAxmol '/axmol.ps1')
    Invoke-Expression "$scriptPath $args"
} else {
    Invoke-Expression "axmol $args"
}
You run it as: ./axmol build -p win32 -c

[halx99]

not sure, I recommend always use latest axmol cmdline tools

[halx99]

that why i'm not copy tools, build.ps1, setup.ps1 to isolated project

[rh101]

not sure, I recommend always use latest axmol cmdline tools

That's a fair point. The example script above can perhaps be just in the wiki, with a note explaining that using such a script may mean not running the latest version of the axmol tools.

[aismann]

This feature make it really easier to test/add/use new extensions/3rdparty/etc with own projects but also add this to the axmol dev line too (any time later).

[rh101]

@rh101 Maybe a wiki entry for this "new" feature is also usefully?

Added: https://github.com/axmolengine/axmol/wiki/FAQ#q-how-to-add-axmol-as-a-git-submodule

[aismann]

@halx99 One more idea: Is it possible to add a parameter like -update for "updating" an isolated project to "any axmol dev (downloaded) repository" ? At the moment a manual copy/paste of axmol to my own project is needed for updating the axmol stuff.

[halx99]

An isolated project means user may modify engine sources, so I can't provide that parameter

[aismann]

An isolated project means user may modify engine sources, so I can't provide that parameter

I understand. Thanks.