Integer type support

[bear24rw]

Are integer types supported? I only get the expected answer for floats or doubles.

Correct answer with type 'float':

#include <stdio.h>
#include <stdint.h>

#define DTYPE float

extern DTYPE __enzyme_autodiff(void*, DTYPE);
DTYPE square(DTYPE x) {
    return x * x;
}
DTYPE dsquare(DTYPE x) {
    return __enzyme_autodiff(square, x);
}
int main() {
    for(DTYPE i=1; i<5; i++)
        printf("square(%f)=%f, dsquare(%f)=%f\n", (float) i, (float) square(i), (float) i, (float) dsquare(i));
}

square(1.000000)=1.000000, dsquare(1.000000)=2.000000
square(2.000000)=4.000000, dsquare(2.000000)=4.000000
square(3.000000)=9.000000, dsquare(3.000000)=6.000000
square(4.000000)=16.000000, dsquare(4.000000)=8.000000

Incorrect answer with type `int':

#include <stdio.h>
#include <stdint.h>

#define DTYPE int

extern DTYPE __enzyme_autodiff(void*, DTYPE);
DTYPE square(DTYPE x) {
    return x * x;
}
DTYPE dsquare(DTYPE x) {
    return __enzyme_autodiff(square, x);
}
int main() {
    for(DTYPE i=1; i<5; i++)
        printf("square(%f)=%f, dsquare(%f)=%f\n", (float) i, (float) square(i), (float) i, (float) dsquare(i));
}

square(1.000000)=1.000000, dsquare(1.000000)=0.000000
square(2.000000)=4.000000, dsquare(2.000000)=0.000000
square(3.000000)=9.000000, dsquare(3.000000)=0.000000
square(4.000000)=16.000000, dsquare(4.000000)=0.000000

[wsmoses]

Integers are presently assumed to be inactive (and thus don't impact the derivative computation).

Allowing active integers is certainly something worth including as we add support for custom accumulations

I'm curious, is there a bigger application use of integers of the differentiable types you have in mind (I haven't been searching, but haven't seen one offhand so it would be helpful to understand).

[bear24rw]

Integers are presently assumed to be inactive (and thus don't impact the derivative computation).

I see, is this something that is "easily" switched on or is it a bigger feature that needs implementation?

I'm curious, is there a bigger application use of integers of the differentiable types you have in mind (I haven't been searching, but haven't seen one offhand so it would be helpful to understand).

My situation is I have a large existing algorithm that I would like to differentiate. The algorithm uses a mix of types and it would be ideal if I could differentiate it "as is".

I wonder if re-writing the types to floats/doubles inside of Enzyme would be a possible solution?

[wsmoses]

Re active integers, I think three things would need to be added:

• Active integers as distinct from inactive integers added to type analysis [in particular differing TBAA parsing]
• A new enzyme_activeint metadata for the calling convention (the way you're passing Enzyme's default calling convention explicitly assumes integers as inactive since the default case of passing a size of an array is more common and thus makes more sense as the default).
• Adjoints for integer operations [add, etc -- most of which can copy the floating variants [fadd], with things like sqrt obviously not needing implementation]

A codebase that uses a mix of types may not actually use active integers, even if it uses integers as part of the computation. The reason for this is a float => integer cast representing a round/floor correctly has a derivative of 0 (or technically a delta function). Thus I'm a bit curious more some more info (also feel free to email me wmoses@mit.edu if that's preferable / you don't want to share globally on github).

Re your last question, rewriting the types to float/double, however, should probably just work.

[bear24rw]

Thanks for the details, I will pursue the second option of converting all types to doubles and see if that works for my applications.

[wsmoses]

Re-opening this issue to preserve active integer type support as being on the radar. I (and several other contributors) likely won't have time to add such support until after the SuperComputing conference deadline, but hopefully can return to this afterwards.

[wsmoses]

Moreover, it should be relatively easy to add support via LLVM's existing fixed point intrinsics (https://llvm.org/docs/LangRef.html#fixed-point-arithmetic-intrinsics).

The question then for your case would be much simpler, assuming you can modify your code to cause those intrinsics to be emitted.