perf: use `@strided` to reduce code duplication

avik-pal commented 2 months ago

Operations Optimized

batchnorm
activation gradient
mean_var

Cleaner Implementations -- TODO

strided cannot handle all array types what to do about that?
- We manually check for can_strided. Too much code duplication. We can reuse the @strided macro code and redefine maybe_strided to not error for arrays not supported by StridedView

TODOs

groupnorm
activation
bias_activation
does Enzyme work natively with strided? my guess would be yes

codecov[bot] commented 2 months ago

Codecov Report

Attention: Patch coverage is 55.55556% with 44 lines in your changes missing coverage. Please review.

Project coverage is 77.89%. Comparing base (c185f04) to head (50931d7).

Files	Patch %	Lines
src/impl/batchnorm.jl	36.36%	42 Missing :warning:
src/impl/common_ops.jl	92.30%	2 Missing :warning:

:exclamation: There is a different number of reports uploaded between BASE (c185f04) and HEAD (50931d7). Click for more details.

HEAD has 10 uploads less than BASE
| Flag | BASE (c185f04) | HEAD (50931d7) | |------|------|------| ||37|27|

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #139 +/- ## ========================================== - Coverage 83.93% 77.89% -6.04% ========================================== Files 37 37 Lines 1867 1814 -53 ========================================== - Hits 1567 1413 -154 - Misses 300 401 +101 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

avik-pal commented 2 months ago

Seems like hand-coded loops are the best way to go

LuxDL / LuxLib.jl