search

nod-ai / iree-amd-aie

IREE plugin repository for the AMD AIE accelerator
Apache License 2.0
69 stars 30 forks source link

Only output bo needs to be synced from device after result is available #849

Closed dezhiAmd closed 1 month ago

dezhiAmd commented 1 month ago

Issue description: In function iree_hal_xrt_direct_command_buffer_dispatch, all buffers are trying to sync from the device regardless of it is buffer for holding input or output. This may impact performance.

Solution: Use member allowed_usage and memory_type in struct iree_hal_buffer_t to decide whether the buffer is for input or output. Currently all input buffer are created here

Assumption: output buffer will never use IREE_HAL_BUFFER_USAGE_MAPPING flag since mapping can be extremely expensive, use limited hardware resources, introduce data hazards, and synchronize host and device execution. But input buffer cannot avoid using this flag

makslevental commented 1 month ago

Currently all input buffer are created here

This is not correct, they are created in many places and the flags are set in many places, such as in runtime/src/iree-amd-aie/driver/xrt/direct_allocator.cc

makslevental commented 1 month ago

There is no buffer created in function iree_hal_xrt_allocator_query_buffer_compatibility which is the link point to.

Please reread my comment again.

makslevental commented 1 month ago

@dezhiAmd please do not resolve conversations without making requested changes. That is our policy.

makslevental commented 1 month ago

@dezhiAmd please also do not delete comments since it completely distorts the discussion.

dezhiAmd commented 1 month ago

@makslevental , Which test/application do you use to test runtime/src/iree-amd-aie/driver/xrt/cts/matmul_dispatch_test.cc?

dezhiAmd commented 1 month ago

At this point, I am not looking for PR approval, I am looking for answers for one question: How to test code in runtime/src/iree-amd-aie/driver/xrt/cts/matmul_dispatch_test.cc? Please share the knowledge if you have it

makslevental commented 1 month ago

@makslevental , Which test/application do you use to test runtime/src/iree-amd-aie/driver/xrt/cts/matmul_dispatch_test.cc?

ctest -R driver/xrt/cts

makslevental commented 1 month ago

At this point, I am not looking for PR approval, I am looking for answers for one question: How to test code in runtime/src/iree-amd-aie/driver/xrt/cts/matmul_dispatch_test.cc? Please share the knowledge if you have it

Please stop re-requesting reviews then. That is not the purpose of review requests. It's sufficient to simply ask the question.