BasicInferer (also simple or single) for no batching
DynamicBatchingInferer generates and memoizes execution plans for each unique batch size
FixedBatchingInferer which subdivides into preconfigured and pregenerated plans.
14 adds DirectInferer which works like DynamicBatchingInferer but skips the memoization step, instead relying on internal concretization for a runtime cost.
All these names are terrible and we should fix them to be more sensible/understandable.
Currently the inferers are named as follows:
BasicInferer(alsosimpleorsingle) for no batchingDynamicBatchingInferergenerates and memoizes execution plans for each unique batch sizeFixedBatchingInfererwhich subdivides into preconfigured and pregenerated plans.14 adds
DirectInfererwhich works likeDynamicBatchingInfererbut skips the memoization step, instead relying on internal concretization for a runtime cost.All these names are terrible and we should fix them to be more sensible/understandable.