Open zero9178 opened 3 months ago
Yeah, I think the current FuncConversion is only designed to work with the full HAL. To use the inline HAL we'll need a mode for the conversion to switch to only generating the synchronous function without any of the full HAL types (fences/devices/etc).
Given a simple example such as:
imported using
iree-turbine
yields the follow IR:This compiles perfectly fine with e.g.
iree-compile test.mlir --iree-hal-target-backends=llvm-cpu -o /dev/null
but fails when trying to usehal_inline
viairee-compile test.mlir --iree-hal-target-backends=llvm-cpu -o /dev/null --iree-execution-model=inline-dynamic
with the following error:As far as I can tell, the issue stems from the existence of the
hal.fence.create
operation which necessitates thehal.device.get
operation which the conversion pass tohal_inline
does not expect as input. Rather it expects the stream dialect and friends which it'd then convert to a synchronous inline implementation that does not need fences that are usually created byStreamToHAL
conversion.The reason the
hal.fence.create
operation is in the IR is due to thetorch-iree-func-conversion
pass: https://github.com/iree-org/iree/blob/d2dd9e23eff96787a66871239c32c627fc3da51b/compiler/plugins/input/Torch/InputConversion/FuncConversion.cpp#L472-L478 It creates a synchronous wrapper using manually created fences. This is seemingly fundamentally incompatible with the conversion pass tohal_inline
unless the latter were to be capable of removing fences (which does not seem trivial in general).