Closed r1c7 closed 5 years ago
When using integral, only after softmax, value in heatmap stands for possibility, before that, the value doesn't have any semantics. In fact, if you print the direct output of network, there are many negatives, but it doesn't matter. @crsailing
If we use heatmap of groundtruth to test, using the integral after softmax, we cannot get the accurate label position.
@crsailing The GT heatmap you assumed, in fact, is totally not the same with prediction of the network. Try to print the direct output of network, and see what the heatmap looks like. By the way, the integral is a new method, not related to onehot/gaussian heatmap, so from this perspective, you cannot assume a GT heatmap, applying integral on it, and expect getting accurate position. It doesn't make any sense.
Hi, if the output is not like a gaussian heatmap, how do you train the network with both heatmap and intergral loss, namely I1 in your paper. The output should be something like a gaussian heatmap since you supervise the network with the gaussian heatmap before the intergral operation?
@chaneyddtt Hi, in practice, we train the network using gaussian loss (H1) and finetune with integral loss(l1)
For integral regression methods (I1, I2, I3, and their multi-stage versions), the network is pre-trained only using heat map loss (thus their H versions) and then, only integral loss is used. We found this training strategy working slightly better than training from scratch using both losses.
Ok, thanks for clarifying.
@lck1201 Hi, is the heatmap on the right column of Figure 2 in the your paper(https://arxiv.org/pdf/1711.08229v1.pdf) ouput from network or softmax?
@Ericjiejie Hi, please email @JimmySuen . I am not quite sure.
Ok, thanks .But I don't know if he will return my email. The reason why I ask is because you say that the output of the network is different from that of GT Heatmap(you talk to @r1ch88), so I think the heatmap on Figure 2 comes from softmax.
I guess it's the output of softmax, beacause only the value after softmax represents possibility.
I agree with you. I have another question. If the output of the network is not like Gaussian heatmap, even very different, why does the author use GT Heatmap for supervision? Will adding this supervision not affect network training?
@lck1201 Hi, where is the I1 loss(heatmap loss and joint loss) in your code? I want to know the whole process of the integral version(I1), because I have some problems to look for in the code.In detail, I don’t understand the heatmap supervision part and want to know what the output of the network is.
I checked it, but there is no heatmap loss, only joint loss.
We only consider one keypoint. If the heatmap is [[0. , 0. , 0. , 0. , 0. , 0. , 0. ], [0. , 0. , 0. , 0. , 0. , 0. , 0. ], [0. , 0. , 0. , 0. , 0. , 0. , 0. ], [0. , 0.1 , 0. , 0. , 0. , 0. , 0. ], [0.2 , 0.3 , 0.2 , 0.2 , 0. , 0. , 0. ], [0.1 , 0.4 , 0.05, 0. , 0. , 0. , 0. ], [0.2 , 0.2 , 0.2 , 0.2 , 0. , 0. , 0. ]]
After using the softmax, the normalized heatmap become [[0.01935992, 0.01935992, 0.01935992, 0.01935992, 0.01935992, 0.01935992, 0.01935992], [0.01935992, 0.01935992, 0.01935992, 0.01935992, 0.01935992, 0.01935992, 0.01935992], [0.01935992, 0.01935992, 0.01935992, 0.01935992, 0.01935992, 0.01935992, 0.01935992], [0.01935992, 0.02139602, 0.01935992, 0.01935992, 0.01935992, 0.01935992, 0.01935992], [0.02364626, 0.02613315, 0.02364626, 0.02364626, 0.01935992, 0.01935992, 0.01935992], [0.02139602, 0.0288816 , 0.02035252, 0.01935992, 0.01935992, 0.01935992, 0.01935992], [0.02364626, 0.02364626, 0.02364626, 0.02364626, 0.01935992, 0.01935992, 0.01935992]] .
So integral regression result will be affected by the value 0.01935992 which is calculated using softmax.