LuxDL / Lux.jl

Elegant & Performant Scientific Machine Learning in Julia
https://lux.csail.mit.edu/
MIT License
456 stars 54 forks source link

Added layer for periodic inputs #626

Closed nicholaskl97 closed 2 months ago

nicholaskl97 commented 2 months ago

Sometimes when we're using physics-informed neural networks (PINNs), we want to solve a PDE on a periodic domain. Common practice is currently to include loss terms enforcing the periodic boundary conditions. The layer added in this PR would structurally enforce periodicity in certain input dimensions by replacing those dimensions $x$ with $\sin(2\pi x / period)$ and $\cos(2\pi x / period)$. This enforces periodicity in a smooth and exact manner at the cost of increasing the effective dimensionality of the input by the number of periodic dimensions. Hopefully, that isn't too penalizing thanks to the scaling of neural nets to higher dimensions.

nicholaskl97 commented 2 months ago

Do we want to go with a different naming? Dense and Scale while common have strong precedence in other frameworks. Periodic is a very general name that would clash quite frequently.

How about SineCosinePeriodicEmbedding? Is that too verbose? Is it more descriptive? It leaves open the possibility of other future periodic embeddings.

avik-pal commented 2 months ago

I like PeriodicEmbedding, we can later add a field to the struct which can take different options like SineCosine, etc.

codecov[bot] commented 2 months ago

Codecov Report

Attention: Patch coverage is 80.00000% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 81.30%. Comparing base (faf1b13) to head (89edbf2). Report is 26 commits behind head on main.

Files Patch % Lines
src/layers/basic.jl 80.00% 2 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #626 +/- ## ========================================== - Coverage 87.42% 81.30% -6.12% ========================================== Files 43 47 +4 Lines 2250 2348 +98 ========================================== - Hits 1967 1909 -58 - Misses 283 439 +156 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.