How to add `negative_slope` to `leaky_relu` activation of pytorch backend?

[EmbraceLife]

I have managed to add LeakyReLU for both keras and pytorch end (see code below). And I want to add an argument alpha in keras or negative_slope in pytorch (equivalent, I guess) to this activation.

I could add alpha to keras LeakyReLU, but failed to add it to pytorch. I wonder when user needs to set value for alpha or negative_slope, how can they do it if we don't make the argument accessible. That's why I want to make the argument available in kur. Or is it that most cases we don't need to change alpha or negative_slope so there is no point of adding them?

Below is the source code I have changed to make LeakyReLU and alpha possible, but not negative_slope for pytorch. Could you check it for me and shed some light on how to add negative_slope for pytorch? Thanks

    def _parse(self, engine):
        """ Parse the layer.
        """

        if not isinstance(self.args, dict):
            self.type = self.args
        else:
        # if alpha not exist or empty as None, default value is 0.3
            self.type = self.args['name']
        if self.type == 'leakyrelu':

            if 'alpha' in self.args and self.args['alpha'] is not None:
                self.alpha = self.args['alpha']
            else:
                self.alpha = 0.3

    ###########################################################################
    def _build(self, model):
        """ Create the backend-specific placeholder.
        """
        backend = model.get_backend()
        if backend.get_name() == 'keras':

            import keras.layers as L            # pylint: disable=import-error

            if self.type != "leakyrelu":
                yield L.Activation(
                    'linear' if self.type == 'none' or self.type is None \
                        else self.type,
                    name=self.name,
                    trainable=not self.frozen
                )
            # if advanced activation in keras, like LeakyReLU
            else:
                yield L.LeakyReLU(alpha=self.alpha)

        elif backend.get_name() == 'pytorch':

            import torch.nn.functional as F     # pylint: disable=import-error
            func = {
                'relu' : F.relu,
                'tanh' : F.tanh,
                'sigmoid' : F.sigmoid,
                'softmax' : F.log_softmax,
                'leakyrelu' : F.leaky_relu
            }.get(self.type.lower())
            if func is None:
                raise ValueError('Unsupported activation function "{}" for '
                    'backend "{}".'.format(self.type, backend.get_name()))

            def connect(inputs):
                """ Connects the layer.
                """
                assert len(inputs) == 1
                return {
                    'shape' : self.shape([inputs[0]['shape']]),
                    'layer' : model.data.add_operation(func)(
                        inputs[0]['layer']
                    )
                }

            yield connect

        else:
            raise ValueError(
                'Unknown or unsupported backend: {}'.format(backend))

[ajsyp]

In the PyTorch section, change this:

func = {
    ...
    'leakyrelu' : F.leaky_relu
    ...
}.get(self.type.lower())

to this:

func = {
    ...
    'leakyrelu' : (lambda x: F.leaky_relu(x, negative_slope=self.alpha))
    ...
}.get(self.type.lower())

Side-note: I would also suggest checking that alpha is not defined when self.type is not leakyrelu.