@oscardssmith brought up the important point today in the meeting that Inference results can depend on the order of queries to inference, since a cached result may be used to resolve a recursive loop more precisely.
Since inference is not always monotonic, we don't even have the guarantee that something being alloc-free when run with a fresh cache will also be alloc-free when it runs after we already have more precise results cached from other function calls.
Some possible solutions:
Compile the function to a @ccallable with GPUCompiler and use that instead of the native Julia function (maybe wrap this in a @generated function to automate the check + compilation)
Annotate the function to disable the use of inference caches to resolve recursive loops (i.e. require that the function must always infer as if no cached results were available)
P.S. A closely-related issue is that our pipeline-parity guarantees with GPUCompiler vs. Base Julia are pretty weak right now. Under optimization pipeline flags, code coverage, etc. it's quite easy for the two to fall out of sync, in which case we lose soundness.
@oscardssmith brought up the important point today in the meeting that Inference results can depend on the order of queries to inference, since a cached result may be used to resolve a recursive loop more precisely.
Since inference is not always monotonic, we don't even have the guarantee that something being alloc-free when run with a fresh cache will also be alloc-free when it runs after we already have more precise results cached from other function calls.
Some possible solutions:
@ccallablewith GPUCompiler and use that instead of the native Julia function (maybe wrap this in a@generatedfunction to automate the check + compilation)P.S. A closely-related issue is that our pipeline-parity guarantees with GPUCompiler vs. Base Julia are pretty weak right now. Under optimization pipeline flags, code coverage, etc. it's quite easy for the two to fall out of sync, in which case we lose soundness.