search

patrick-kidger / equinox

Elegant easy-to-use neural networks + scientific computing in JAX. https://docs.kidger.site/equinox/
Apache License 2.0
2.12k stars 142 forks source link

MLP "depth" parameter doc #780

Open tyagi-ankit opened 4 months ago

tyagi-ankit commented 4 months ago

It says:

depth: The number of hidden layers, including the output layer. For example, depth=2 results in an network with layers: [Linear(in_size, width_size), Linear(width_size, width_size), Linear(width_size, out_size)].

Shouldn't it be "excluding"? From the example it looks like that.

patrick-kidger commented 4 months ago

Hmm, based on my reading that looks correct. In the example we have (a) the input layer, and then (b) two more layers. Corresponding to depth=2.

There's never been a great job at standardising terminology in the field here, however...

tyagi-ankit commented 4 months ago

Thanks for the answer Patrick. It's confusing to me. I thought input and output layers are never counted as hidden layers. To me, input layer has no parameters. So, when it says "The number of hidden layers, including the output layer", I read (a) 2 hidden layers, and then (b) one output layer.

But thanks for the clarification.