While working on TensorRT BYOC, I found that the TensorRT runtime requires the weight tensor to be passed as a constant at compile time. So we need to use BindParams before partitioning, but currently it results in the following IR:
This PR adds an option to OperatorFusor, to allow keeping bound constants in the original position rather than lifting them to parameters. This is used when OperatorFusor is used by the FuseOpsByPattern pass.
While working on TensorRT BYOC, I found that the TensorRT runtime requires the weight tensor to be passed as a constant at compile time. So we need to use
BindParams
before partitioning, but currently it results in the following IR:Instead, what we need for BYOC is:
This PR adds an option to
OperatorFusor
, to allow keeping bound constants in the original position rather than lifting them to parameters. This is used whenOperatorFusor
is used by theFuseOpsByPattern
pass.@Hzfengsy @tqchen