openxla / stablehlo

Backward compatible ML compute opset inspired by HLO/MHLO
Apache License 2.0
411 stars 112 forks source link

Convert stablehlo/testdata to use the Check dialect #1218

Open burmako opened 1 year ago

burmako commented 1 year ago

It would be good to update the conformance suite to use the Check dialect, so that the tests are more easily aligned with IREE.

sdasgup3 commented 1 year ago

@GleasonK

Following the suggestion here in this issue and the fact that VHLO does not support the check ops, does it make sense add the custom_call wrapper around the check ops just before serialization and accordingly unwrap after de-serialization.

GleasonK commented 1 year ago

This is the reason custom_call is currently used. I think a lot of these questions require a design document and discussions with stakeholders that we won't have a chance to do in the near future (unless we reshuffle a bit).

There is a tradeoff no matter how we go about it:

Option 1, Test files use custom_call, and provide passes to meet consumer requirements. Under this design we require consumers to call these passes in their test infra. This has the benefit that not all consumers need any of these transforms. Ex1: Any backend that converts these tests into an MLIR dialect that may optimize away constants will need to call these passes before running their test.

Option 2, Test files use check dialect and have optimization_barriers (#1220) embedded. Under this design, some consumers may need to write transformations to remove these bits they don't want. Ex1: StableHLO has no folders so if a backend consumes StableHLO they now need to support optimization_barrier just for these tests. Ex2: VHLO doesn't support check dialect so we need to convert them away into custom calls for serialization as you mention.