In order to lower the bar of technical expertise to using the O3DE as well as to reduce build times of users not modifying the core Engine, the O3DE as an SDK feature was proposed. The primary goal of this features is to allow users to use and distribute the engine with pre-built libraries and applications, that can allow for working on a Game or Simulation Project without the need to build the engine from source code. The SDK will contain the Source Assets, configuration files, cmake build scripts, library and binary artifacts in a layout that is relocatable and transferable to other users. Furthermore it provides a mechanism for registering multiple engines on a user machine that can be associated with O3DE Projects that are also registered.
This effort involves a fundamental shift in the way developers use the engine.
Previously Lumberyard developers had to store their Game Projects within the Engine Source directory and build their game from there. This inherently make the engine mutable and therefore non-distributable.
Organizations, studios and users were not able to share their Game Projects among each other without also providing that specific build of engine as well.
In this manner the Lumberyard Engine was the consumer of Game Project. This will be called the engine-centric workflow.
To accomplish the goal of using O3DE as an SDK, there needs to be support for the O3DE Game Projects to instead use the engine instead. Therefore a new workflow will be added which will be called the "project-centric" workflow.
Summary of Work Already Done
The following is a summary of work that has already been completed for the O3DE as an SDK effort before the public release of O3DE
Provided users with a much smaller package to download and start working with the O3DE engine. Previously, distributing Lumberyard involved making a package that contained the build artifacts for every project within the engine source directory, but also additional binaries and scripts (/Tools) that weren't needed to run the Engine.
Included in this package was standalone applications that had to be run to download all the 3rdParty libraries (SetupAssistant), as well as additional Game Projects, sometimes containing legacy file structure, along with their large source assets. Frankly most of these things were not needed by users creating their own project.
Provided support for storing Projects and Gems externally from the Engine.
This opens up the possibility of distributing Projects and Gems individually and can drive the adoption of a community repo of shared Projects and Gems.
Reduction of build times for developers and content creators.
Builds take minutes instead of hours as only the Project code and any Project registered gems need to compile. The slowest part of interacting with the Engine should be link time.
Allow users to create their own SDK layouts of the Engine that can distribute to their user/customer/employee base(Windows).
Provided python package for registering Engines, Projects, Gems and Templates,
Also provided commands for creating projects and gems out of templates and transform projects and gems to templates.
Provided a way to associate an Engine with a Project, to allow a project to not be tied to an engine location on a specific machine.
Different users are able to install their engine anywhere on the local machine and use that registered engine with the shared Game/Simulation Project as long as it contains the same engine identifier.
The engine installer(O3DE_installer) effort will wrap the work proposed in this RFC to provide the user with an application that can install the O3DE SDK as well as any necessary dependencies(cmake, python, etc...).
Furthermore the Project Manager will wrap the O3DE python package script provided by this effort in a GUI that allows users to manage their Projects, Gems and Templates.
Summary of Upcoming Work
This provides a short summary of the upcoming work of the O3DE as an SDK effort in the next milestone.
Add support for users to create an SDK layout for Linux
Add support for users to create an SDK layout for MacOS
Add support for users to create an SDK layout for iOS
Add support for users to create an SDK layout for Android.
Add support for creating an SDK layout out of a Project-Centric Source-Engine workflow.
The SDK layout should contain both the Project and Engine build artifacts along with the Source Assets, build and configuration scripts and other metadata.
Investigate the ability to create a hybrid SDK that can both can contain host-platform Tools(Linux, MacOS, Windows) and non-host platform SDK layout files(Android and iOS).
This will help ease the development process of iOS and Android developers wanting to work with a single O3De SDK to build their Android or iOS game
Provide scripts for users to create creating a Project Game Release layout, which includes the processed assets, configuration files, loadable archive files and the launcher applications.
This is to provide a mechanism for a Game/Simulation Team to provide a bundled all the necessary artifacts needed for their end-user's to run their game/simulation.
Business Need
Feedback from Users of Lumberyard have indicated that getting from download to a usable Editor is quite long and causes users to lose interest in the product. With the O3DE as an SDK effort, it allows Studios, Game Designer, Artist and Novices to use O3DE without the barrier of needing to perform long builds of the Engine in order to author a game or simulation.
There is also a need for consistent workflow for teams to create the layout needed to build & deploy their final game builds with the appropriate processed assets and configuration files. Moreover the O3DE as an SDK effort opens up possibilities for online communities where organizations and individuals can share their own Projects, Gems and Templates. It provides a mechanism for Gem authors to offer pre-built versions of their gems to users reducing iteration time for users wanting to experiment with a gem.
Feature Breakdown
So far the O3DE as an SDK effort is already through 2 Milestones.
The task for in those milestones are listed in the Appendix Below.
The High-Level work needed for Milestone 3 is as follows
Make sure an SDK layout can be created on MacOS for all supported configurations and that the SDK can be used to build a project
Create an SDK Layout for Linux
Validates an SDK layout for Linux is supported for all supported configurations and that the SDK can be used to build a project
Create an SDK Layout for iOS
Add support for an SDK layout that can be used to build an O3DE Project targeting iOS. This task doesn't imply that the SDK contains the MacOS tools needed to iterate on a Project such as the Editor, AssetProcessor
Create an SDK Layout for Android
Implement changes to allow an SDK layout targeting the Android Platform to be created(at the very least on Windows). Also this task would need to verify that a GameLauncher can be built and run on an Android device using the SDK
Support SDK Layout Creation in a Project-Centric Source-Engine workflow
This kind of SDK layout can be used to iterate on the project, but it is not meant to be distributed to the end-user(Player). This is to facilitate a workflow where a team of developers can work in on a Project with the Engine-Source code and then provide a pre-built SDK for their Artist/Game Designers to speed up iteration
Add support for creating a Project Release layout with the bundled assets.
This involves adding a CMake step that will allow users to not only build the all Project's build artifacts, but also process the necessary assets for the running the project, bundling those assets using the Asset Bundler and copying the build artifacts with the bundled assets to a "Project Game/Simulation release" layout folder. This layout will contain all the files that O3DE can provide that will run the Game on an end-user machine.
Investigate the creation of a Hybrid SDK Layout
An SDK for non-host platforms such as Android and iOS, will also need a subset of the SDK from the host platforms of Windows, Linux and MacOS. In order to process Assets for Android, an Asset Processor is needed on Windows, but not necessarily the libraries needed to build Windows applications
Add Support for creating an SDK Layout for Monolithic builds
Users will be able to create SDK layout an Engine configured to build monolithic binaries. The 'release' configuration will be officially supported, while 'debug/profile' configuration could work, but will not receive official support status. Monolithic builds are normally used in the final released game executable to avoid the need to load shared libraries at runtime and to optimize the size of the game package
Optional Work
This is additional task that can make it easier for teams/studios to develop a game or simulation product using the SDK, but not strictly required.
Optional Task
Description
Support SDK Layout Creation in a Project-Centric SDK-Engine workflow
This is a special case of having studio or team being able to take a pre-built Engine SDK layout, using that to create a project, and then using the Project-Centric workflow to then create an SDK layout containing the project build artifacts + Engine build artifacts. Such an approach is meant for distribution within a game team or studio working on a project. This isn't strictly required as it is expected that most game teams would need to modify the engine
Dependencies
The remaining work in the O3DE as an SDK effort would allow users to use the Project Manager application on Linux and Mac for project management using a UI.
This will also unlock the ability for the Installer team effort or some other initiative to create packaging or installer for installing the O3DE SDK on Linux or Mac.
Scope
The following table details what works is in-scope for the O3DE as an SDK effort and what is out of scope
In-Scope
Registering projects, gems, and engines
Querying of information from registration manifest files(o3de manifest)
Creation of templates from a directory structure and instantiation of templates to location on the filesystem
Creation of an SDK Layout using CMake.
Creation of a "Project Release Layout" that contains all the artifacts needed to run a game on platform
Out of Scope
This effort only covers the work to support users (Game Studio, individual, O3D Foundation) being able create an SDK layout. How the O3D Foundation delivers an pre-built SDK to users is part of the Installer work being done by the sig-release group
Currently there exist an installer executable for Windows, Any plans for an installer for Mac(perhaps a .dmg) or a package for Linux is outside the scope of this effort. Linux especially so, since the package system is tied to the Distribution(ie. Yum, DPKG, apt, pacman), with differing package formats(.deb, .tar.xz, .rpm, etc...)
Also out of scope is the creation of a GUI tool for configuration and registration of Projects, Gems and Templates. The Project Manager application(O3DE.exe) which is owned by the Project Manager work being done by the sig-release group. That work wraps the O3DE python package which provides this functionality.
Fixing any issues related the AssetBundler if it has issues gathering necessary Asset Dependencies for a project release layout. Any AssetBundler issues will be tagged to the sig-content group
Questions and Concerns
IDE Debugging(Visual Studio/Xcode)
The ability to debug SDK applications in popular IDEs is an important aspect of driving users to use the Project-Centric SDK workflow over using the Project-Centric Engine Source workflow.
Particular the Editor and AssetProcessor applications would benefit if they appeared as targets within the CMake Build System. This would allow applications like Visual Studio and Xcode to have a target that would allow users to debug those targets.
Other tools for debugging SDK applications such as WinDBG, lldb and gdb don't need to have such an IDE target are not a concern, since the users using these tools have the expectation that they know how to find and attach an application using these tools from the Command Line.
### Questions
Where will debugging symbols be stored for the pre-built binaries of the SDK Layout?
Also how will users obtain debugging symbols for downloaded copies of the SDK layout?
On Windows PDB files are needed for debugging. How can users obtain them?
The PDBs could be uploaded an O3DE the Symbol Server or there could be a separate downloadable package with just the .pdb files within it.
### Concerns
The availability of Source Code debugging of the SDK Layout.
The commits in which an SDK layout was built against would be useful, for other users to determine which git commit hash is needed to provide source level debugging of the SDK.
The MSVC debugger, lldb, gdb, and WinDBG all support specifying a path to source code for debugging. So users would need to know the matching commit to get accurate debugging.
Hybrid Layout
Support for a Hybrid SDK layout of having the host platform build artifacts with non-host platforms build artifacts are needed for full use of the SDK for non-host platforms, but how this is tackled is not defined yet.
By Hybrid SDK, what is meant is distributing an SDK layout on Windows that can be used to build an O3DE Project for Android and therefore needs the Android build artifacts(shared library .so and static library .a files), but also requires the Windows Editor.exe and AssetProcessor.exe binaries in order to author content for the O3DE Project and to process the Assets for the Android platform.
### Issues
Because O3DE uses the CMake build generator, only one platform at a time can be configured and built into any specific build folder.
This means that to build an SDK layout for a platform such as Android or iOS, requires at least building a subset of the SDK layout for the host platform(Windows, Linux, Mac) and adding that to the layout, followed, by building the Android or iOS platform itself and copying it to the layout.
As these are isolated build solutions, the process of creating the SDK layout is no longer a single INSTALL step, but multiple for each platform.
Furthermore the INSTALL step also supports a single platform at a time, so in-order to create a complete install layout for a single platform the INSTALL step must be run three times
### Concerns
Having a hybrid layout also implies an increase in the overall Install Layout Size.
It has been seen on Linux that just building an SDK Layout for Debug is upwards of 100GiB.
Therefore if for some reason in the future a Linux SDK Layout, that contained support for Android was built, it could be quite large a package for download.
Mitigation plans should be drafted to try to reduce the binary size build artifacts with the SDK, while still allowing for debug-ability of the SDK layout for users.
Linux Cross-Distribution Support
Linux sits in an area, where there are multiple different distributions with different packaging systems, tweaks to the kernel, and OS specific APIs that makes distributing an SDK layout created on one OS not viable for other OS outside of that family.(Arch vs Debian vs RedHat vs Fedora vs etc...).
All the 3rdParty libraries are are copied into the SDK Layout is copied from the runtime dependencies of the build targets within the Engine. Using only the 3rdParty libraries that are part of the 3rdParty package system should work in most cases, but depending on how old the Linux kernel is for that distribution, some newer core libraries might not be available.
Testing will only occur on Ubuntu 20.04 LTS
Any testing of an SDK layout created on Linux will be done only for Ubuntu 20.04 distribution.
The community will be empowered to see how well the O3DE as an SDK works on other platforms.
### Issues
Because Linux supports multiple distributions, building an SDK with one distribution, might result in libraries that users versions of glibc or libstdc++ that are newer than the versions that part of the "supported distribution list" of O3DE.
Chances are building O3DE as an SDK on ArchLinux, will not work on Ubuntu 20.04 due to ArchLinux having newer versions of kernels and core libraries installed
None of this work covers an SDK Layout on an ARM architecture. The Install layout logic doesn't take any architecture into account, so that running the INSTALL step to create a layout should work regardless of running on ARM Linux, ARM Mac, ARM Windows.
The issue however is that none of those platforms have 3rdParty libraries built for them. and therefore the build step would fail
Mac ARM64/x86_64 Compatibility
With the introduction of the Mac M1, it appears that Apple is moving towards using ARM architecture for all their platforms going forward.
The Mac M1 also comes with a Rosetta Translation Environment that is used to translate applications which only contain x86_64 instructions to ARM64.
This means that Mac M1 machines can support running ARM64 programs natively and x86_64 programs through emulation, while older Mac x86_64 machines can only support running x86_64 programs.
Xcode does support creating a Universal Binary MacOS binary on both architectures, therefore it is possible to target build a universal binary release of O3DE on any of the MacOS machines at that moment.
### Questions
Should universal binary be made for MacOS or should only the newer ARM64 architecture be targeted when building an SDK layout?
Universal binaries would be larger in size increasing the SDK layout overall size.
### Issues
All of the Current 3rdParty libraries for MacOS is built with x86_64, not ARM64.
In order to get full native support on newer Mac M1 machines, those libraries would need to be recompiled for the ARM64 Architecture
Engine Versioning
Version of the O3D Engine itself is an unsolved problem. This is exacerbated even more when using a upgrading a pre-built SDK to a newer version.
Currently the engine specifies the version in the engine.json file as "O3DEVersion", but that value is always "0.0.0.0" at the moment.
Furthermore the project.json does not specify which version of the engine it desires to use.
### Questions
How should the o3de python registration scripts account for versioning registering a project?
Should users be able to specify a single value, a range, can it accept non-numeric values?
Should CMake prevent configuration when a project is associated with an engine out side of the version range the project specified?
### Concerns
As Game Development is an iterative process that involves modification of the engine, there will be Game Teams that build a new Engine SDK based on those changes.
In those cases, the range of valid version a Game Project specifies cannot be based on the commit id as those might not be in sequential order depending on the source control system being used.
The versioning scheme will not take the source control system into account and therefore cannot account for ranges of commits.
Stakeholders
Group
Rationale
sig-build
Would want to know the changes to CMake build system to support creating a SDK and Project Release Layout. Also would need to be appraised of any build pipeline changes needed for building a layout on Jenkins
sig-core
The O3DE as an SDK work requires changes to core systems(Engine/Project Registration, Gem System, Template creation system). Also the group responsible for implementing the O3DE as an SDK effort
sig-platform
The O3DE an SDK effort is targeting multiple OS which will require additional platform specific files to be added to the layout. Furthermore deploying a GameLauncher to specific platform requires bespoke knowledge of how to run an application on specific devices such as Android and iOS
sig-release(Installer and Project Manager efforts)
Currently working on an installer for O3DE that allows users to not have to download O3DE repo and build the engine from source. Also maintains the ProjectManager application which exposes project and gem management as it related to the a registered engine. Builds on top of the work done by the O3DE as an SDK effort
sig-testing
The default workflow of creating an SDK creates a layout an SDK Layout without the internal AutomatedTesting project. Being able to run the python test scripts located in the AutomatedTesting project would be useful in validating the Editor workflow performs the same as in a source engine.
Appendix
Completed Work
The following is the work already completed on the O3DE as an SDK Effort
Milestone 1 - External Project and Gem Support, Manifest Registration system (Completed)
Milestone 1 Task
Description
Project-centric workflow
In this workflow, the project ingests the engine through CMake instead of the old Engine-centric workflow where the reverse was true. Now CMake can be run using the Project Root as the Source Directory
External Projects
Refactor of the engine source code and build scripts to be able to specify and use a project located outside of the Engine Source directory.
External Gems
Addition of CMake hooks(LY_EXTERNAL_SUBDIRS) and manifest logic to allow users to register a Gem externally to Engine Source directory.
Project and Gem Templates
Creation of the new Project and Gem Templates that uses new layout different from 1.X to organize project's and gems and work with the CMake build system and Settings Registry
Manifest Registration System
o3de python package scripts that can register gem, project, templates and external subdirectories paths with either a global user manifest(~/.o3de_manifest.json), a engine manifest(engine.json) or a project manifest(project.json). The Manifest allows projects to identify an engine to use in a system independent manner. This allows external projects on one user machine to user a different location for the engine as other users.
Template Creation System
A different subset of o3de python package scripts that can take a registered template and transform it to a Gem, Project or general folder with placeholders replaced
Milestone 2 - Windows Install Layout Creation, Gem Naming Convention and Dependency Tracking (Completed)
Milestone 2 Task
Description
CMake Install Process - Windows
This involves the marking of build targets, folders and files within CMake to determine which files should be copied over to the install layout
Generation of the Install Layout CMakeLists.txt
The install layout requires CMakeLists.txt files that defines the pre-built libraries and executables, that can be referenced by an O3DE Project for use when building that project. Any target that is build from source using the Source Engine, needs to be available as an IMPORTED target that references the build artifact in the SDK Engine
Ensure that non-build systems files that are needed to use the Engine as relocatable SDK is copied to the Install Layout
To actually use the Engine in any reasonable manner, a multitude of additional files are needed. The <engine-root>/Assets folder needs to be copied to the Install layout in order to access the Core Source Assets. Each Gem that is registered with the Engine may contain also an Assets folder exposing Source Assets that available when that Gem is enabled. The <engine-root>/scripts folder is needed to copy over the O3DE Python Package Scripts which is used for project management and Asset Bundling. The <engine-root>/Registry folder is needed to copy the default settings that the SDK Engine uses when running. Furthermore each Gems Registry folder needs to be copied over for using Gem specific settings. Finally the <engine-root>/python/get_python.bat scripts are need to make sure the SDK layout is able to download a copy of python 3.7 that isolated for use with the Engine and to install the site-packages used by the engine(assimp, pyside2, pybind11, o3de)
Locating Projects Gem Modules to load from the Install layout Applications
The Pre-Built applications that come with the install layout, such as the Asset Processor and Editor needs to be able to locate the list of Gem Modules to that the Project desires to load
Gem Naming Convention
Having a Gem Naming Convention eases not only the logic to enable/disable gem by tools(o3de package enable_gem.py/disable_gem.py), but also allows users to no longer need to specify the specific build target when enabling/disabling a gem. The Gem Name can just be used for the gem configuration instead of the CMake target. This also reduces the simplifies on the o3de python package scripts logic to parse CMake scripts to determine which Gem are enabled. Furthermore having a consistent naming convention allows Gem Authors to easily segment which types of Application a Gem is used for into "Variants". Variants are aliases which aggregate 0 or more CMake build targets within a Gem to expose a consistent name. The initial list of variants are "Clients", "Servers", "Tools" and "Builders". "Clients" variants are used within the Game Launcher. "Server" variants are used within the Server Launcher. "Tools" variants are used in tooling applications such as the Editor, MaterialEditor and ShaderManagementConsole. "Builders" variants are used in asset processing applications such as the AssetProcessor(Batch) and AssetBuilder
Overview
In order to lower the bar of technical expertise to using the O3DE as well as to reduce build times of users not modifying the core Engine, the O3DE as an SDK feature was proposed. The primary goal of this features is to allow users to use and distribute the engine with pre-built libraries and applications, that can allow for working on a Game or Simulation Project without the need to build the engine from source code. The SDK will contain the Source Assets, configuration files, cmake build scripts, library and binary artifacts in a layout that is relocatable and transferable to other users. Furthermore it provides a mechanism for registering multiple engines on a user machine that can be associated with O3DE Projects that are also registered.
This effort involves a fundamental shift in the way developers use the engine.
Previously Lumberyard developers had to store their Game Projects within the Engine Source directory and build their game from there. This inherently make the engine mutable and therefore non-distributable.
Organizations, studios and users were not able to share their Game Projects among each other without also providing that specific build of engine as well.
In this manner the Lumberyard Engine was the consumer of Game Project. This will be called the engine-centric workflow.
To accomplish the goal of using O3DE as an SDK, there needs to be support for the O3DE Game Projects to instead use the engine instead. Therefore a new workflow will be added which will be called the "project-centric" workflow.
Summary of Work Already Done
The following is a summary of work that has already been completed for the O3DE as an SDK effort before the public release of O3DE
Included in this package was standalone applications that had to be run to download all the 3rdParty libraries (SetupAssistant), as well as additional Game Projects, sometimes containing legacy file structure, along with their large source assets. Frankly most of these things were not needed by users creating their own project.
This opens up the possibility of distributing Projects and Gems individually and can drive the adoption of a community repo of shared Projects and Gems.
Builds take minutes instead of hours as only the Project code and any Project registered gems need to compile. The slowest part of interacting with the Engine should be link time.
Also provided commands for creating projects and gems out of templates and transform projects and gems to templates.
Different users are able to install their engine anywhere on the local machine and use that registered engine with the shared Game/Simulation Project as long as it contains the same engine identifier.
The engine installer(O3DE_installer) effort will wrap the work proposed in this RFC to provide the user with an application that can install the O3DE SDK as well as any necessary dependencies(cmake, python, etc...).
Furthermore the Project Manager will wrap the O3DE python package script provided by this effort in a GUI that allows users to manage their Projects, Gems and Templates.
Summary of Upcoming Work
This provides a short summary of the upcoming work of the O3DE as an SDK effort in the next milestone.
The SDK layout should contain both the Project and Engine build artifacts along with the Source Assets, build and configuration scripts and other metadata.
This will help ease the development process of iOS and Android developers wanting to work with a single O3De SDK to build their Android or iOS game
This is to provide a mechanism for a Game/Simulation Team to provide a bundled all the necessary artifacts needed for their end-user's to run their game/simulation.
Business Need
Feedback from Users of Lumberyard have indicated that getting from download to a usable Editor is quite long and causes users to lose interest in the product. With the O3DE as an SDK effort, it allows Studios, Game Designer, Artist and Novices to use O3DE without the barrier of needing to perform long builds of the Engine in order to author a game or simulation.
There is also a need for consistent workflow for teams to create the layout needed to build & deploy their final game builds with the appropriate processed assets and configuration files. Moreover the O3DE as an SDK effort opens up possibilities for online communities where organizations and individuals can share their own Projects, Gems and Templates. It provides a mechanism for Gem authors to offer pre-built versions of their gems to users reducing iteration time for users wanting to experiment with a gem.
Feature Breakdown
So far the O3DE as an SDK effort is already through 2 Milestones.
The task for in those milestones are listed in the Appendix Below.
The High-Level work needed for Milestone 3 is as follows
Milestone 3 - Linux, Mac, iOS and Android Install Layout Support, Project Release Layout Creation, Hybrid SDK Investigation
Optional Work
This is additional task that can make it easier for teams/studios to develop a game or simulation product using the SDK, but not strictly required.
Dependencies
The remaining work in the O3DE as an SDK effort would allow users to use the Project Manager application on Linux and Mac for project management using a UI.
This will also unlock the ability for the Installer team effort or some other initiative to create packaging or installer for installing the O3DE SDK on Linux or Mac.
Scope
The following table details what works is in-scope for the O3DE as an SDK effort and what is out of scope
How the O3D Foundation delivers an pre-built SDK to users is part of the Installer work being done by the sig-release group
Linux especially so, since the package system is tied to the Distribution(ie. Yum, DPKG, apt, pacman), with differing package formats(.deb, .tar.xz, .rpm, etc...)
The Project Manager application(O3DE.exe) which is owned by the Project Manager work being done by the sig-release group.
That work wraps the O3DE python package which provides this functionality.
Any AssetBundler issues will be tagged to the sig-content group
Questions and Concerns
IDE Debugging(Visual Studio/Xcode)
The ability to debug SDK applications in popular IDEs is an important aspect of driving users to use the Project-Centric SDK workflow over using the Project-Centric Engine Source workflow.
Particular the Editor and AssetProcessor applications would benefit if they appeared as targets within the CMake Build System. This would allow applications like Visual Studio and Xcode to have a target that would allow users to debug those targets.
Other tools for debugging SDK applications such as WinDBG, lldb and gdb don't need to have such an IDE target are not a concern, since the users using these tools have the expectation that they know how to find and attach an application using these tools from the Command Line.
### Questions
Also how will users obtain debugging symbols for downloaded copies of the SDK layout?
The PDBs could be uploaded an O3DE the Symbol Server or there could be a separate downloadable package with just the .pdb files within it.
### Concerns
The commits in which an SDK layout was built against would be useful, for other users to determine which git commit hash is needed to provide source level debugging of the SDK.
The MSVC debugger, lldb, gdb, and WinDBG all support specifying a path to source code for debugging. So users would need to know the matching commit to get accurate debugging.
Hybrid Layout
Support for a Hybrid SDK layout of having the host platform build artifacts with non-host platforms build artifacts are needed for full use of the SDK for non-host platforms, but how this is tackled is not defined yet.
By Hybrid SDK, what is meant is distributing an SDK layout on Windows that can be used to build an O3DE Project for Android and therefore needs the Android build artifacts(shared library .so and static library .a files), but also requires the Windows Editor.exe and AssetProcessor.exe binaries in order to author content for the O3DE Project and to process the Assets for the Android platform.
### Issues
Because O3DE uses the CMake build generator, only one platform at a time can be configured and built into any specific build folder.
This means that to build an SDK layout for a platform such as Android or iOS, requires at least building a subset of the SDK layout for the host platform(Windows, Linux, Mac) and adding that to the layout, followed, by building the Android or iOS platform itself and copying it to the layout.
As these are isolated build solutions, the process of creating the SDK layout is no longer a single INSTALL step, but multiple for each platform.
Furthermore the INSTALL step also supports a single platform at a time, so in-order to create a complete install layout for a single platform the INSTALL step must be run three times
### Concerns
It has been seen on Linux that just building an SDK Layout for Debug is upwards of 100GiB.
Therefore if for some reason in the future a Linux SDK Layout, that contained support for Android was built, it could be quite large a package for download.
Mitigation plans should be drafted to try to reduce the binary size build artifacts with the SDK, while still allowing for debug-ability of the SDK layout for users.
Linux Cross-Distribution Support
Linux sits in an area, where there are multiple different distributions with different packaging systems, tweaks to the kernel, and OS specific APIs that makes distributing an SDK layout created on one OS not viable for other OS outside of that family.(Arch vs Debian vs RedHat vs Fedora vs etc...).
All the 3rdParty libraries are are copied into the SDK Layout is copied from the runtime dependencies of the build targets within the Engine. Using only the 3rdParty libraries that are part of the 3rdParty package system should work in most cases, but depending on how old the Linux kernel is for that distribution, some newer core libraries might not be available.
Testing will only occur on Ubuntu 20.04 LTS
Any testing of an SDK layout created on Linux will be done only for Ubuntu 20.04 distribution.
The community will be empowered to see how well the O3DE as an SDK works on other platforms.
### Issues
Chances are building O3DE as an SDK on ArchLinux, will not work on Ubuntu 20.04 due to ArchLinux having newer versions of kernels and core libraries installed
The issue however is that none of those platforms have 3rdParty libraries built for them. and therefore the build step would fail
Mac ARM64/x86_64 Compatibility
With the introduction of the Mac M1, it appears that Apple is moving towards using ARM architecture for all their platforms going forward.
The Mac M1 also comes with a Rosetta Translation Environment that is used to translate applications which only contain x86_64 instructions to ARM64.
This means that Mac M1 machines can support running ARM64 programs natively and x86_64 programs through emulation, while older Mac x86_64 machines can only support running x86_64 programs.
Xcode does support creating a Universal Binary MacOS binary on both architectures, therefore it is possible to target build a universal binary release of O3DE on any of the MacOS machines at that moment.
### Questions
### Issues
In order to get full native support on newer Mac M1 machines, those libraries would need to be recompiled for the ARM64 Architecture
Engine Versioning
Version of the O3D Engine itself is an unsolved problem. This is exacerbated even more when using a upgrading a pre-built SDK to a newer version.
Currently the engine specifies the version in the engine.json file as "O3DEVersion", but that value is always "0.0.0.0" at the moment.
Furthermore the project.json does not specify which version of the engine it desires to use.
### Questions
### Concerns
In those cases, the range of valid version a Game Project specifies cannot be based on the commit id as those might not be in sequential order depending on the source control system being used.
Stakeholders
Also would need to be appraised of any build pipeline changes needed for building a layout on Jenkins
Also the group responsible for implementing the O3DE as an SDK effort
Furthermore deploying a GameLauncher to specific platform requires bespoke knowledge of how to run an application on specific devices
such as Android and iOS
Also maintains the ProjectManager application which exposes project and gem management as it related to the a registered engine.
Builds on top of the work done by the O3DE as an SDK effort
Being able to run the python test scripts located in the AutomatedTesting project would be useful in validating the Editor workflow performs
the same as in a source engine.
Appendix
Completed Work
The following is the work already completed on the O3DE as an SDK Effort
Milestone 1 - External Project and Gem Support, Manifest Registration system (Completed)
Milestone 2 - Windows Install Layout Creation, Gem Naming Convention and Dependency Tracking (Completed)
<engine-root>/Assetsfolder needs to be copied to the Install layout in order to access the Core Source Assets. Each Gem that is registered with the Engine may contain also anAssetsfolder exposing Source Assets that available when that Gem is enabled. The<engine-root>/scriptsfolder is needed to copy over the O3DE Python Package Scripts which is used for project management and Asset Bundling. The<engine-root>/Registryfolder is needed to copy the default settings that the SDK Engine uses when running. Furthermore each GemsRegistryfolder needs to be copied over for using Gem specific settings. Finally the<engine-root>/python/get_python.batscripts are need to make sure the SDK layout is able to download a copy of python 3.7 that isolated for use with the Engine and to install the site-packages used by the engine(assimp, pyside2, pybind11, o3de)