Closed boazcogan closed 3 years ago
The description of the leaky ReLU section doesn't match the image. current wording:
The Leaky ReLU is similar to the regular ReLU, but instead of the output being 0.0 below 𝑥=0.0 it is ever so slightly above 0.0
This suggest that the output of the function is always positive, maybe instead the wording can be something along the lines of:
The Leaky ReLU is similar to the regular ReLU, but instead of the output being 0.0 below 𝑥=0.0 it is weighted with some small value, typically 0.01x.
The description of the leaky ReLU section doesn't match the image. current wording:
This suggest that the output of the function is always positive, maybe instead the wording can be something along the lines of:
The Leaky ReLU is similar to the regular ReLU, but instead of the output being 0.0 below 𝑥=0.0 it is weighted with some small value, typically 0.01x.