Asset Converter 2.0 - Githubissues

devshgraphicsprogramming commented 2 years ago

Description

Need an asset converter that works.

Description of the related problem

Current converted not suitable for Vulkan, does not:

care about usage flags (will merge buffers that shouldn't be merged)
[x] promote images to usable formats

Perf/Efficiency:

wrong assumption that the same Asset Type wil only be encountered at the same level of the Directed Acyclic Graph of Assets
[x] cannot "stream upload images" yet

Ergonomics and Correctness:

templates everywhere
does not use compute shader to produce mip-maps

Solution proposal

As input take a sorted dynamic_refctd_array<IAsset> of root assets + conversion options argument (current SParams)
Do not use templates and perform a Depth First Search to explore the DAG starting from the roots
Figure out what Buffers and Images need to be de-aliased due to format promotion (promotion occurs in the view)
Use unordered_map<cpu_asset_type_and_metadata,gpu_asset_type_and_metadata> to store all assets to convert, the key metadata will probably just be the promoted format if any, the value metadata will contain modifiers like additional usages etc.
Then commence converting assets, starting form the bottom of the logical stack
ALWAYS If you can find the cached asset conversion with the correct metadata (usages, etc.) then use that, otherwise proceed with steps 7 onwards, at each step check all dependents are ready and figure out if can recover from error before creating anything
Create Buffers, bucketed by usage, depending on options trim and merge them into large suballocated buffers (we already do this)
Create Images without any merging
Figure out the heaps and memory types each IGPUBuffer and IGPUImage need, and whether they need to be dedicated, bucket the allocations by that
Allocate Device Memory and bind it to the created Buffers and Images (allows for suballocation)
Fill Buffers and Images with data and convert if needed (remember about de-aliased promoted copies!)
Create Image Views, Buffer Views and Acceleration Structures while remembering to promote and adjust any offsets after any promotion or packing/merging (probably should modified Hash Cons here, try and find duplicate views)
Create Samplers with Hash Consing
Create Descriptor Set Layouts with Hash Consing
Create Pipeline Layouts with Hash Consing
Create Shaders (maybe with Hash Consing, since we can has and compare the source code)
Create Specialized Shaders (with modified Hash Consing - basically compare for equality after hotswapping shaders)
Create Compute and RenderpassIndependentPipelines (with modified Hash Consing)
Create Graphics Pipelines (with modified Hash Consing)
Create MeshBuffers (with modified Hash Consing)
Create Meshes (with modified Hash Consing)
Return an dynamic_refctd_array<IReferenceCounted> wrapped in a utility struct

Special concerns:

Use the provided logger extensively
[x] Stream and convert (if needed due to promotion) the Image regions via Staging Buffer, no intermediate GPU Buffers with all data
[x] Work on completely immutable assets (const everywhere)
[x] Ignore ICPUBuffers used to feed ICPUImage conversions

Additional context

Hash certain objects to not create needless copies

You can hash samplers, and descriptor set layouts, pipeline layouts (via hash consing).

This will reduce the number of samplers and layouts created (important cause VK has limits on this shit).

You can apply this regardless of if the assetManager has been provided as an external cache (also useful to make it work on hashes of the samplers and layouts).

CPU Asset Usages are hints

When creating a Buffer or Image (or in the future ImageView) remember that usages are hints, you need to patch them up during DAG traversal (add usages which are implied by the DAG if they're not there already).

You cannot however, "narrow down the usages" if a user has explicitly stated that a Buffer has a usage, it must have that usage.

This allows us to handle promotion and de-aliasing better, as de-aliased buffers and images are an extra copy and not the original, hence they need not have any of the usages explicitly enumerated by the original, only the implicit ones from the views that actually use them.

Controlling Options

General Options:

[ ] whether to attempt to patch up assets with failed dependants (i.e. just disable a vertex attribute, if vertex buffer failed creation)
[ ] whether to allow the "shrinking" of data ranges (shrink image by removing subresource ranges not used by any view, shrink buffers by removing any ranges not used by view or binding... while respecting alignments of all of them) for non-root assets (being root implies whole range is used)
[ ] whether to disallow the "packing" of buffer ranges non-root assets
[ ] whether to allow the "packing" of buffer ranges for root assets (disables the disallowance of packing non-root buffers)
[ ] whether to allow format promotion
[ ] whether to treat a promotion as a failure to create a root asset (return nullptr for the associated retval slot)
[ ] whether to allow de-aliasing (creating different copies of an underlying Buffer or Image for different views of the same underlying) during promotion

Patching-up BufferViews and VertexAttribute Bindings

After converting an entire buffer range due to a promotion, the relative offset may change due to shrinking and/or having a larger format.

This is also a concern for strides for vertex buffer bindings in graphics pipelines and acceleration structures.

Erfan-Ahmadi commented 2 years ago

Additional Comments based on talk with @devshgraphicsprogramming

Pack buffers with similar usages together into seperate classes (same for images), new BufferAPI allows suballocating buffers from a big IDeviceMemoryAllocation
We convert assets in several passes : for example 1. batch splitting (figure out all the dependants) and assign to their classes 2. Create GPU resources 3. ????