search

o3de / o3de.org

The O3DE website
Other
86 stars 159 forks source link

[DOCS] Working with Atom & RPI standalone #1742

Closed chaterpaul closed 1 year ago

chaterpaul commented 2 years ago

Hi,

I would like to use the atom render engine as a standalone entity. The documentation in this link: https://www.o3de.org/docs/atom-guide/dev-guide/rpi/working-with-scene-and-rendering-pipeline/

Is not sufficient to initialize the system as calling :

RPI::Scene::CreateScene(sceneDesc);

leads to a nullptr exception due to RPISystemInterface::Get() returning nullptr

Is there further documentation or an example on how to setup and use Atom as standalone. The AtomSampleViewer application is too complex and has alot of intialization dependent on json files and environments. A simple application example which opens a window and creates scene and a camera would be very instructive.

Regards

Paul

FiniteStateGit commented 2 years ago

Requesting additional information from @o3de/sig-graphics-audio.

santorac commented 2 years ago

The primary way to use Open 3D Engine and the Atom renderer is to build a game project in O3DE and package it using the built-in Game Launcher application. AtomSampleViewer is the best example of how to build something like a standalone application that uses Atom, but it still has many dependencies on core parts of O3DE (memory management, math, IO, asset processing, etc).

Can you share more detail about what your end goals are?

chaterpaul commented 2 years ago

Hi,

We have an engineering application which runs as a desktop application on windows which displays 3d graphics in a viewport. The Atom renderer looks great and using it would improve the performance of our application. I would like to see if its possible to create a simple application which will open a window which would host it which I could then expand upon as a proof of concept. I have downloaded and built the AtomSampleViewer application, but am struggling with extracting the components I need and what components are necessary to build my simple application. The initialization of the system by looking for .setreg files in the Registry directory and the order and pieces of code, directories, setup files etc. I need is difficult to work out, and I haven’t found the appropriate documentation yet. Consuming just the necessary parts of O3DE I don’t think would be a problem for us.

Thanks for your help

Paul

From: santorac @.> Sent: 29 June 2022 17:50 To: o3de/o3de.org @.> Cc: Paul Chater @.>; Author @.> Subject: Re: [o3de/o3de.org] [DOCS] Working with Atom & RPI standalone (Issue #1742)

WARNING: This email originated from outside of the organization. DO NOT click links, open attachments, or respond unless you recognize the sender and know the content is safe.

The primary way to use Open 3D Engine and the Atom renderer is to build a game project in O3DE and package it using the built-in Game Launcher application. AtomSampleViewer is the best example of how to build something like a standalone application that uses Atom, but it still has many dependencies on core parts of O3DE (memory management, math, IO, asset processing, etc).

Can you share more detail about what your end goals are?

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/o3de/o3de.org/issues/1742*issuecomment-1170232371__;Iw!!F1Q1IbZmrAg!HFVUYgrkq-lHfJElVoXzvoIl4TEUt_TjVry6mUmo862jQe558yQe4Gqi7pPfs732K_xZ8aAjTrTaDioGJ1HdaMZeOe0$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AKDMBPOU5MG5QI4HZ3MWQXDVRR5ELANCNFSM5ZT3SXQA__;!!F1Q1IbZmrAg!HFVUYgrkq-lHfJElVoXzvoIl4TEUt_TjVry6mUmo862jQe558yQe4Gqi7pPfs732K_xZ8aAjTrTaDioGJ1Hd3ZK9SpU$. You are receiving this because you authored the thread.Message ID: @.**@.>>

This email, including any attachments, may contain confidential and/or proprietary information intended only for the use of the recipient. If you are not the intended recipient, any distribution, copying, or use of this email or its attachments is prohibited. If you received this email in error, please reply to the sender immediately and delete this message and any copies.

Bentley Systems has taken all reasonable steps to ensure that this communication is free from viruses, data corruption, and unauthorized alteration. Bentley Systems does not accept liability for any damages that may be incurred as a result of this or any communication by email

[https://cdn2.webdamdb.com/310th_sm_UnR3pO7k0ir0.jpg?1616176329]

rgba16f commented 2 years ago

Hi Paul,

I'm the person that wrote a lot of the logic that controls O3DE initializing Atom so I know the most, it's been a couple of years though so please excuse me as I re-dig up the information you need.

Most of the marshalling of the startup is controlled from the BootstrapSystemComponent and the RPISystemComponent, both of which are created during application initialization for O3DE. The code is here o3de/Gems/Atom/Bootstrap/Code/Source/BootstrapSystemComponent.cpp and o3de/Gems/Atom/RPI/Code/Source/RPI.Private/RPISystemComponent.cpp. The RPISystemComponent is activated before the BootstrapSystemComponent.

There is a lot of code the BootstrapSystemComponent to deal with Atom being run in a Qt application (the editor) or the GameLauncher because that controls who creates the window Atom renders into. In the editor Qt creates the window and in the GameLauncher Atom creates the window (using the NativeWindow class).

After the window is created there are various steps to create the default scene, the default window context, and the default render pipeline.

After that the bootstrap ticks the viewport context for the GameLauncher.

Hopefully that is enough for you to start, please feel free to follow up with any more details you need. I can also be reached on the o3de discord as rgba16f.

Please note that Jul 4th is a holiday in the USA so I may not get back to you until after that.

rgba16f

chaterpaul commented 2 years ago

Hi,

Thanks for replying. I will look through the two .cpp files you list below which will no doubt lead me to some more questions but I would like to start with understanding the auxiliary files included with the atomsampleviewer which participate in its initialization.

  1. cmake_dependencies.*.setreg. This file gets loaded from the application start method. It looks to load various dlls and call their ::Initialize & ::Activate methods, is there any documentation on the contents/structure of this file you can provide, its location relative to the .exe, what the dlls you are loading are doing etc.
  2. The directories, Passes, Shaders, ShaderLib, ShaderVariants. Is there any documentation on the different type of files contained within, .azl, .shader, .pass, .azasset, such that I can understand where and how these participate in the setup of the atom renderer.
  3. Is it possible to not require the assetprocessor, by loading the necessary assets in the main process, all of the geometry we need to display will be read out of our file and then we can create entities, lights, meshes, materials etc. in the atom renderer.

I currently have the framework of an application mainly extracted from the atomsampleviewer for learning purposes and have added various pieces of code found from your documentation to see if I can create a window then host the atom renderer in it.

I get a fair few errors such as :

================================================================= RPI Utils: Trace::Error D:/src/o3de/Gems/Atom/RPI/Code/Source/RPI.Public/RPIUtils.cpp(341): 'struct AZ::Data::AssetId __cdecl AZ::RPI::GetShaderAssetId(const class AZStd::basic_string<char,struct AZStd::char_traits,class AZStd::allocator> &,bool)' RPI Utils: Could not load critical shader [Shaders/auxgeom/auxgeomobjectlit.azshader] RPI Utils: ================================================================== RPI Utils:

When trying to activate an rhiscene (I think this is to do with not using the asset processor)

I can create a ::NativeWindow and a ::WindowContext and initialize it however when I try and create a renderpipeline for the windowContext I get a SwapChain crash, as below, this leads me to think I am missing something with the initialization, hence the questions above to aid my understanding of the necessary auxiliary files.

================================================================== System: Trace::Assert D:/src/o3de/Gems/Atom/RPI/Code/Source/RPI.Public/Pass/Specific/SwapChainPass.cpp(78): (29116) 'void __cdecl AZ::RPI::SwapChainPass::SetupSwapChainAttachment(void)' System: PassTemplate used to create SwapChainPass must have an InputOutput called SwapChainOutput System: ------------------------------------------------ System: D:\src\o3de\Gems\Atom\RPI\Code\Source\RPI.Public\Pass\Specific\SwapChainPass.cpp (80) : AZ::RPI::SwapChainPass::SetupSwapChainAttachment System: D:\src\o3de\Gems\Atom\RPI\Code\Source\RPI.Public\Pass\Specific\SwapChainPass.cpp (97) : AZ::RPI::SwapChainPass::BuildInternal System: D:\src\o3de\Gems\Atom\RPI\Code\Source\RPI.Public\Pass\Pass.cpp (1237) : AZ::RPI::Pass::Build System: D:\src\o3de\Gems\Atom\RPI\Code\Source\RPI.Public\Pass\Pass.cpp (1432) : AZ::RPI::Pass::ManualPipelineBuildAndInitialize System: D:\src\o3de\Gems\Atom\RPI\Code\Source\RPI.Public\RenderPipeline.cpp (112) : AZ::RPI::RenderPipeline::InitializeRenderPipeline System: D:\src\o3de\Gems\Atom\RPI\Code\Source\RPI.Public\RenderPipeline.cpp (100) : AZ::RPI::RenderPipeline::CreateRenderPipelineForWindow System: d:\src\Viewer\Viewer\ViewerApplication.cpp (1491) : Viewer::CreateDefaultCamera System: d:\src\Viewer\Viewer\ViewerApplication.cpp (1533) : Viewer::RunGameCommon

I have added myself to discord.com, I sent a friend request to you from Buzzieeeeee#9917 so we can continue this conversation there.

Thanks again for your assistance.

Paul

From: rgba16f @.> Sent: 02 July 2022 01:16 To: o3de/o3de.org @.> Cc: Paul Chater @.>; Author @.> Subject: Re: [o3de/o3de.org] [DOCS] Working with Atom & RPI standalone (Issue #1742)

WARNING: This email originated from outside of the organization. DO NOT click links, open attachments, or respond unless you recognize the sender and know the content is safe.

Hi Paul,

I'm the person that wrote a lot of the logic that controls O3DE initializing Atom so I know the most, it's been a couple of years though so please excuse me as I re-dig up the information you need.

Most of the marshalling of the startup is controlled from the BootstrapSystemComponent and the RPISystemComponent, both of which are created during application initialization for O3DE. The code is here o3de/Gems/Atom/Bootstrap/Code/Source/BootstrapSystemComponent.cpp and o3de/Gems/Atom/RPI/Code/Source/RPI.Private/RPISystemComponent.cpp. The RPISystemComponent is activated before the BootstrapSystemComponent.

There is a lot of code the BootstrapSystemComponent to deal with Atom being run in a Qt application (the editor) or the GameLauncher because that controls who creates the window Atom renders into. In the editor Qt creates the window and in the GameLauncher Atom creates the window (using the NativeWindow class).

After the window is created there are various steps to create the default scene, the default window context, and the default render pipeline.

After that the bootstrap ticks the viewport context for the GameLauncher.

Hopefully that is enough for you to start, please feel free to follow up with any more details you need. I can also be reached on the o3de discord as rgba16f.

Please note that Jul 4th is a holiday in the USA so I may not get back to you until after that.

rgba16f

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/o3de/o3de.org/issues/1742*issuecomment-1172794149__;Iw!!F1Q1IbZmrAg!FT-lbH22R8zToQ2vSKaXFrb4COKKpdL-C6T15ytb2btuvA3qINEpk36kpckOBEXBnICgU-uoAXrgsQBx5byXpWLu9bc$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AKDMBPONFPIQPAYQNXGUKE3VR6C5PANCNFSM5ZT3SXQA__;!!F1Q1IbZmrAg!FT-lbH22R8zToQ2vSKaXFrb4COKKpdL-C6T15ytb2btuvA3qINEpk36kpckOBEXBnICgU-uoAXrgsQBx5byXs_rzkY0$. You are receiving this because you authored the thread.Message ID: @.**@.>>

This email, including any attachments, may contain confidential and/or proprietary information intended only for the use of the recipient. If you are not the intended recipient, any distribution, copying, or use of this email or its attachments is prohibited. If you received this email in error, please reply to the sender immediately and delete this message and any copies.

Bentley Systems has taken all reasonable steps to ensure that this communication is free from viruses, data corruption, and unauthorized alteration. Bentley Systems does not accept liability for any damages that may be incurred as a result of this or any communication by email

[https://cdn2.webdamdb.com/310th_sm_UnR3pO7k0ir0.jpg?1616176329]

lemonade-dm commented 2 years ago

Hi, Thanks for replying. I will look through the two .cpp files you list below which will no doubt lead me to some more questions but I would like to start with understanding the auxiliary files included with the atomsampleviewer which participate in its initialization. 1. cmake_dependencies.*.setreg. This file gets loaded from the application start method. It looks to load various dlls and call their ::Initialize & ::Activate methods, is there any documentation on the contents/structure of this file you can provide, its location relative to the .exe, what the dlls you are loading are doing etc. 2. The directories, Passes, Shaders, ShaderLib, ShaderVariants. Is there any documentation on the different type of files contained within, .azl, .shader, .pass, .azasset, such that I can understand where and how these participate in the setup of the atom renderer. 3. Is it possible to not require the assetprocessor, by loading the necessary assets in the main process, all of the geometry we need to display will be read out of our file and then we can create entities, lights, meshes, materials etc. in the atom renderer. I currently have the framework of an application mainly extracted from the atomsampleviewer for learning purposes and have added various pieces of code found from your documentation to see if I can create a window then host the atom renderer in it. I get a fair few errors such as : ================================================================= RPI Utils: Trace::Error D:/src/o3de/Gems/Atom/RPI/Code/Source/RPI.Public/RPIUtils.cpp(341): 'struct AZ::Data::AssetId cdecl AZ::RPI::GetShaderAssetId(const class AZStd::basic_string<char,struct AZStd::char_traits,class AZStd::allocator> &,bool)' RPI Utils: Could not load critical shader [Shaders/auxgeom/auxgeomobjectlit.azshader] RPI Utils: ================================================================== RPI Utils: ================================================================== When trying to activate an rhiscene (I think this is to do with not using the asset processor) I can create a ::NativeWindow and a ::WindowContext and initialize it however when I try and create a renderpipeline for the windowContext I get a SwapChain crash, as below, this leads me to think I am missing something with the initialization, hence the questions above to aid my understanding of the necessary auxiliary files. ================================================================== System: Trace::Assert D:/src/o3de/Gems/Atom/RPI/Code/Source/RPI.Public/Pass/Specific/SwapChainPass.cpp(78): (29116) 'void cdecl AZ::RPI::SwapChainPass::SetupSwapChainAttachment(void)' System: PassTemplate used to create SwapChainPass must have an InputOutput called SwapChainOutput System: ------------------------------------------------ System: D:\src\o3de\Gems\Atom\RPI\Code\Source\RPI.Public\Pass\Specific\SwapChainPass.cpp (80) : AZ::RPI::SwapChainPass::SetupSwapChainAttachment System: D:\src\o3de\Gems\Atom\RPI\Code\Source\RPI.Public\Pass\Specific\SwapChainPass.cpp (97) : AZ::RPI::SwapChainPass::BuildInternal System: D:\src\o3de\Gems\Atom\RPI\Code\Source\RPI.Public\Pass\Pass.cpp (1237) : AZ::RPI::Pass::Build System: D:\src\o3de\Gems\Atom\RPI\Code\Source\RPI.Public\Pass\Pass.cpp (1432) : AZ::RPI::Pass::ManualPipelineBuildAndInitialize System: D:\src\o3de\Gems\Atom\RPI\Code\Source\RPI.Public\RenderPipeline.cpp (112) : AZ::RPI::RenderPipeline::InitializeRenderPipeline System: D:\src\o3de\Gems\Atom\RPI\Code\Source\RPI.Public\RenderPipeline.cpp (100) : AZ::RPI::RenderPipeline::CreateRenderPipelineForWindow System: d:\src\Viewer\Viewer\ViewerApplication.cpp (1491) : Viewer::CreateDefaultCamera System: d:\src\Viewer\Viewer\ViewerApplication.cpp (1533) : Viewer::RunGameCommon I have added myself to discord.com, I sent a friend request to you from Buzzieeeeee#9917 so we can continue this conversation there. Thanks again for your assistance. Paul From: rgba16f @.> Sent: 02 July 2022 01:16 To: o3de/o3de.org @.> Cc: Paul Chater @.>; Author @.> Subject: Re: [o3de/o3de.org] [DOCS] Working with Atom & RPI standalone (Issue #1742) WARNING: This email originated from outside of the organization. DO NOT click links, open attachments, or respond unless you recognize the sender and know the content is safe. … ____ Hi Paul, I'm the person that wrote a lot of the logic that controls O3DE initializing Atom so I know the most, it's been a couple of years though so please excuse me as I re-dig up the information you need. Most of the marshalling of the startup is controlled from the BootstrapSystemComponent and the RPISystemComponent, both of which are created during application initialization for O3DE. The code is here o3de/Gems/Atom/Bootstrap/Code/Source/BootstrapSystemComponent.cpp and o3de/Gems/Atom/RPI/Code/Source/RPI.Private/RPISystemComponent.cpp. The RPISystemComponent is activated before the BootstrapSystemComponent. There is a lot of code the BootstrapSystemComponent to deal with Atom being run in a Qt application (the editor) or the GameLauncher because that controls who creates the window Atom renders into. In the editor Qt creates the window and in the GameLauncher Atom creates the window (using the NativeWindow class). After the window is created there are various steps to create the default scene, the default window context, and the default render pipeline. After that the bootstrap ticks the viewport context for the GameLauncher. Hopefully that is enough for you to start, please feel free to follow up with any more details you need. I can also be reached on the o3de discord as rgba16f. Please note that Jul 4th is a holiday in the USA so I may not get back to you until after that. rgba16f — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/https:/github.com/o3de/o3de.org/issues/1742*issuecomment-1172794149;Iw!!F1Q1IbZmrAg!FT-lbH22R8zToQ2vSKaXFrb4COKKpdL-C6T15ytb2btuvA3qINEpk36kpckOBEXBnICgU-uoAXrgsQBx5byXpWLu9bc$>, or unsubscribe<https://urldefense.com/v3/https:/github.com/notifications/unsubscribe-auth/AKDMBPONFPIQPAYQNXGUKE3VR6C5PANCNFSM5ZT3SXQA;!!F1Q1IbZmrAg!FT-lbH22R8zToQ2vSKaXFrb4COKKpdL-C6T15ytb2btuvA3qINEpk36kpckOBEXBnICgU-uoAXrgsQBx5byXs_rzkY0$>. You are receiving this because you authored the thread.Message ID: @.**@.>> ____ This email, including any attachments, may contain confidential and/or proprietary information intended only for the use of the recipient. If you are not the intended recipient, any distribution, copying, or use of this email or its attachments is prohibited. If you received this email in error, please reply to the sender immediately and delete this message and any copies. Bentley Systems has taken all reasonable steps to ensure that this communication is free from viruses, data corruption, and unauthorized alteration. Bentley Systems does not accept liability for any damages that may be incurred as a result of this or any communication by email [https://cdn2.webdamdb.com/310th_sm_UnR3pO7k0ir0.jpg?1616176329]

Hello @chaterpaul , for Number 1, we are in the progress of adding documentation detailing the Settings Registry and how gem modules are loaded as part of this PR. Here is the link to the documentation section preview: https://deploy-preview-1623--o3deorg.netlify.app/docs/user-guide/settings/settings-registry-developer-documentation/#loading-gems-through-the-settings-registry.

Pretty much the ComponentApplication::LoadDynamicModule function is responsible for reading from the Settings Registry the merged set of settings containing the names of active gems and their modules that should be loaded within the application. The Settings key for the module to load is underneath /O3DE/Gems/<GemName>/Targets/Modules array in the cmake_dependencies.<project-name>.<CMake target name>.setreg file.

For example when running the GameLauncher for a project called "TestProject". The following Settings Registry file will get loaded. cmake_dependencies.testproject.testproject_gamelauncher.setreg. If the Editor is loaded and supplied the project path to "TestProject", then the file of cmake_dependencies.testproject.editor.setreg is loaded. The linked documentation goes into more details about how the cmake_dependencies.*.setreg files are generated and loaded using the Settings Registry specialization system.

Now after the gem modules are loaded, the ModuleManager will go lookup an extern "C" exported symbol of "CreateModuleClass" for each loaded module and invoke that function to create an AZ::Module derived class, which contains functions for reflecting components to Reflection Context(SerializeContext, EditContext, BehaviorContext) and to specify the list of required "System" components(singletons) that should be created after the gem module has finished loading.

The System Components for each loaded module are Activated when the System Entity is activated within the Application::StartCommon function. The Module Manager is listening to the OnEntityActivated EBus event to initialize the gem module required system components in ActivateEntities

antonmic commented 2 years ago

@chaterpaul for your #2: "The directories, Passes, Shaders, ShaderLib, ShaderVariants. Is there any documentation on the different type of files contained within, .azl, .shader, .pass, .azasset, such that I can understand where and how these participate in the setup of the atom renderer." For Passes, the current documentation is under https://www.o3de.org/docs/atom-guide/dev-guide/passes/ You can also learn more about the pass system design with this talk: https://www.youtube.com/watch?v=brLHCsmfr-I&list=PLCQwFpnHSZQgkw7MdNQwagKKKvywt6b80&index=9&ab_channel=Open3DEngine