search

keras-team / keras-core

A multi-backend implementation of the Keras API, with support for TensorFlow, JAX, and PyTorch.
Apache License 2.0
1.27k stars 116 forks source link

Duplicated implementations of silu / swish #695

Closed Frightera closed 1 year ago

Frightera commented 1 year ago

Currently in ops.nn.py there are two identical implementations for silu and swish (which are the same things):

https://github.com/keras-team/keras-core/blob/d40529efb31f6759699318bfb76927505bd95f76/keras_core/ops/nn.py#L181-L226

https://github.com/keras-team/keras-core/blob/d40529efb31f6759699318bfb76927505bd95f76/keras_core/backend/numpy/nn.py#L39-L44

Why just not return one of the implementations, say backend.nn.silu in class Swish? Or it would be better if it is possible to use the export decorator like this:

In (ops.nn.py):

@keras_core_export(["keras_core.ops.silu", "keras_core.ops.nn.silu", "keras_core.ops.nn.swish", "keras_core.ops.swish"])
fchollet commented 1 year ago

Yes, you are right. There is no need for the duplication, we can just export API aliases. I've consolidated everything under silu (since the name predates swish).