About the network training

[lianxxx]

Thanks for your repo! I tried to put this CRFasRNN module in a ResNet based segmentation network but the loss did not decrease. Could you please give any suggestion on this situation, such as how to choose hyper-parameters, how to initialise the CRF layers. I am confused why in the original paper, 1e-13 such a small learning rate was chosen, any reason for it??

Any replies are much appreciated.

[Fettpet]

For me the learning rate of 1e-13 leads to numeric instabilities. I don't know how the authors are able to use these small learning rate. I train it with U-Net and Deeplab v3. I trained it on noisy medical images.

I used the SGD optimizer

U-Net DenseCRF:

1. CNN: epsilon =0,00006; μ = 0,889; λ = 0,037,
2. CRF: epsilon = 0,00002; μ= 0,811; λ = 0,574;
3. Parameter θ α = 170; θ β = 1,569; θ γ = 3,367.

DeepLab DenseCRF:

1. CNN: epsilon = 0,00023, μ= 0,839 λ=0,393,
2. CRF: epsilon = 0,000096, μ= 0,854 λ=0,689
3. Parameter: θ α = 181, θ β = 1,244, θ γ = 1,449

epsilon: learning rate λ: Regularization μ: Momentum

[lianxxx]

Dear author, recently I built a new environment with pytorch 1.7.1 and cuda 11.2, then I compiled with errors. It shows /bin/sh: 1: :/usr/local/cuda/bin/nvcc: not found, but nvcc is there. Do you have any suggestions? Thanks very much.