RuntimeError: a leaf Variable that requires grad is being used in an in-place operation.

[kuangbo]

When I reproduce the BiFPN, a problem occured. Then, I get. Leaf Variable can not use nn.ReLU(), except F.relu(). However, why is the following code okay? https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch/blob/acb2c691e1220448d41527b3e3b5c56206d8fd70/efficientdet/model.py#L143

My code as follows:

class Fusefeature(nn.Module):
    # replace concat with fusefeature
    def __init__(self, e=1e-4):    # concat, nums_input, epsilon, dimension
        super(Fusefeature, self).__init__()
        self.e = e
        self.weight = nn.Parameter(torch.ones(2, dtype=torch.float32), requires_grad=True)
        self.weight_relu = nn.ReLU()
        self.swish = MemoryEfficientSwish() if not onnx_export else Swish()

    def forward(self, x):
        weight = self.weight_relu(self.weight)
        weight = weight / (torch.sum(weight, dim=0) + self.e)
        return self.swish(weight[0] * x[0] + weight[1] * x[1])

Thanks for your attention !

[zylo117]

Try performing relu on self.weight.clone(), instead of self.weight