kostaleonard / great-model-theory

A deep learning library for Scala.
MIT License
2 stars 0 forks source link

Updates to ModelParameters may erroneously overwrite multiple copies #79

Closed kostaleonard closed 1 year ago

kostaleonard commented 1 year ago

The ModelParameter case class currently contains two constructor arguments: name (String) and value (NDArray). If there are two ModelParameters with identical names and values, it is likely that an update to a ModelParameter could erroneously overwrite both. For example, let's say we have two Dense layers with 100 units. If the biases are initialized with name "biases", and they each start with the default value of ones, then the case class equality mechanism would recognize them as the same object, and we would provide the same gradient updates to each even though the updates would be incorrect for one layer.