About 3D Version

[HiphonL]

Hi, Thanks for your nice work. By the way, I want to know if this implementation of 3D version can be directly used for irregular discrete points？

[MiguelMonteiro]

Hi,

This C++ code applies a CRF on a fully connected graph, it is not required that it is an image. That being said the way the graph is defined, the function that calculates the potentials between the nodes, is specific to images in the sense that it uses distance in space as well as the channel values. This can be changed by changing the functions that computes this affinity compute_kernel to make the rest of the code applicable to the new type of graph.

Best,

Miguel

[HiphonL]

Hi,

This C++ code applies a CRF on a fully connected graph, it is not required that it is an image. That being said the way the graph is defined, the function that calculates the potentials between the nodes, is specific to images in the sense that it uses distance in space as well as the channel values. This can be changed by changing the functions that computes this affinity compute_kernel to make the rest of the code applicable to the new type of graph.

Best,

Miguel

Hi Miguel,

Thanks for your reply. I have changed the compute_kernel and made it work according to your suggestions. However, when I trained my model with the CRFasRNN as the top layer, the loss sometime has been computed as "nan", which has never happened on the model without the CRFasRNN layer. Additionally, the “nan”-loss seems to be sensitive to theta_alpha, theta_beta, theta_gamma according to my experiments. Larger these hyper parameters made “nan”-loss more likely. In fact, I don't know for what.

Do you have any idea to solve this problem?

The training records are shown as following: loss: 0.233059, ACC: 0.946615, mIoU:0.660923 loss: 0.316464, ACC: 0.921094, mIoU:0.596161 loss: nan, ACC: 0.935156, mIoU:0.640983 loss: 0.254759, ACC: 0.936888, mIoU:0.649814 loss: nan, ACC: 0.937070, mIoU:0.648452 loss: 0.278317, ACC: 0.938737, mIoU:0.636103 loss: 0.237098, ACC: 0.947292, mIoU:0.688795 loss: 0.330383, ACC: 0.898424, mIoU:0.620089 loss: 0.229100, ACC: 0.942578, mIoU:0.679973

Best, Haifeng

[MiguelMonteiro]

It's very sensitive to those hyper-parameters, you need to see how they relate to the new way you are computing the kernel - "affinity between nodes" to see what makes sense. On one side you will get nan-loss on the other you might get that the layer ins't doing anything.

[HiphonL]

Hi Miguel,

I have gone over the workflow. Now, maybe, I have found the reasons for the nan-loss.

The first is a gradient explosion in the RNN module (in the crf_rnn_layer.py). Due to there is not any activate function for the convolution operation and compatibility transform to restrain the gradient, the gradient explosion is easy to happen with a large iteration. So, can the RNN module be changed into the mode of LSTM to overcome the gradient explosion? or any other idea?

The second one is about the _module.latticefilter. Could you let me know if this function will output nan-value in some cases? e.g., divided by zero, etc.

Kind Regards,

Haifeng

[MiguelMonteiro]

It is not possible to replace the RNN for an LSTM as that is not the point. The RNN is there as an unrolling of the mean-field approximation algorithm for CRF estimation so it doesn't make sense to change it. I think _module.latticefilter will keep working regardless of the input being nan or infinity but you can check yourself by directly feeding tensors into this module. The way to address the issue of exploding gradients is to control the theta hyperparameters to reasonable values. Unfortunately this is hard and possibly application specific.