A library for solving differential equations using neural networks based on PyTorch, used by multiple research groups around the world, including at Harvard IACS.
I saw you implemented the Swish function. As an exercise i implemented the APTx activation https://arxiv.org/abs/2209.06119 , which seems to be a variant of the new mish function that requires less computational power. I tested on the DE given as exercise during the Harvard ComputeFest and it seems to give better results than Swish at low/very low epochs. However more tests are needed before a definitive response.
I saw you implemented the Swish function. As an exercise i implemented the APTx activation https://arxiv.org/abs/2209.06119 , which seems to be a variant of the new mish function that requires less computational power. I tested on the DE given as exercise during the Harvard ComputeFest and it seems to give better results than Swish at low/very low epochs. However more tests are needed before a definitive response.