Any plans to add custom loss functions?

[Simplex-AP]

The python version of MXNet has the capability to define custom loss functions, but this is missing from the Julia version. Are there any plans to add it?

[vchuravy]

Depends on #39 and see my work in #16. The ability to define custom loss functions requires layers written in python and is what inspired me to work on it. I have plans to work on this but it will need to wait till julia v0.5

[pluskid]

Another option which might be easier to implement (though involves quite a lot refactoring but not technically difficult) is the new module system in the mxnet python side. See dmlc/mxnet#1868 for example. With the new module system, a hybrid symbolic and imperative module can be used. The computation graph is built using symbolic nodes, while the loss function is written directly in Python.

Unfortunately, I have no estimate of timeline when I would be able to find enough time to port that to Julia side. Contributions are very welcome of course!

[Arkoniak]

There is a discussion In R branch of MXNet. They propose to use MakeLoss function. I've tried to implement their solution in julia:

• linear regression
• logistic regression

Well, generally speaking it works, but it looks ugly. it would be nice, if anyone with deep knowledge of MXNet could help.

There are few questions:

• Because NN + custom loss layer has loss output as a result, it is not clear, how to calculate other eval metrics during training.
• For the same reason, i had to keep two different neural networks, one for fit, another one for predict. May be there is better solution? Default output layers like SoftmaxOutput and others do not have this problem.
• Even if there is need to keep two different networks, it would be nice to abstract loss layer, for future reuse. I guess it should be bind/simple_bind somehow to base net, but wasn't able to how to use this syntax.