We have this eps parameters because we noticed that the eps value had an affect on how many intervals we needed to use in the integrated gradient for layer norm layers. There's some discussion about it here https://apolloresearchhq.slack.com/archives/C05V1AGUQB1/p1697117599097219, though I think much of it was in person.
Changing this parameter will change the computation that occurs in the layer, so any edge/C that we get out wouldn't be completely valid if we used a value that wasn't the one we trained on. We should just remove this optional argument.
We have this eps parameters because we noticed that the eps value had an affect on how many intervals we needed to use in the integrated gradient for layer norm layers. There's some discussion about it here https://apolloresearchhq.slack.com/archives/C05V1AGUQB1/p1697117599097219, though I think much of it was in person.
Changing this parameter will change the computation that occurs in the layer, so any edge/C that we get out wouldn't be completely valid if we used a value that wasn't the one we trained on. We should just remove this optional argument.