Extension of for-loop range and modification to reproduce pixel ROCAUC as described in README

@byungjae89 Thank you for your great job always!

I made a minor program modification. In my testing, this modification had no impact on detection performance at MVTec.

[code before]

                # calculate distance matrix
                dist_matrix_list = []
                # feat_gallery = feat_gallery[:(feat_gallery.shape[0] // 100) * 100]
                print('feat_gallery.shape =', feat_gallery.shape)
                for d_idx in range(feat_gallery.shape[0] // 100):  # original, not add one
                    # print('[d_idx*100:d_idx*10+100] = [%d:%d]' % (d_idx*100, d_idx*100+100))
                    dist_matrix = torch.pairwise_distance(feat_gallery[d_idx * 100:d_idx * 100 + 100], test_feat_map)
                    dist_matrix_list.append(dist_matrix)
                dist_matrix = torch.cat(dist_matrix_list, 0)
                print('dist_matrix.shape =', dist_matrix.shape)

[debug before]

feat_gallery.shape = torch.Size([15680, 256, 1, 1])
dist_matrix.shape = torch.Size([15600, 256, 56])
feat_gallery.shape = torch.Size([3920, 512, 1, 1])
dist_matrix.shape = torch.Size([3900, 512, 28])
feat_gallery.shape = torch.Size([980, 1024, 1, 1])
dist_matrix.shape = torch.Size([900, 1024, 14])

[measure before]

bottle ROCAUC: 0.972
bottle pixel ROCAUC: 0.562
cable ROCAUC: 0.848
cable pixel ROCAUC: 0.654
capsule ROCAUC: 0.897
capsule pixel ROCAUC: 0.638
carpet ROCAUC: 0.928
carpet pixel ROCAUC: 0.663
grid ROCAUC: 0.471
grid pixel ROCAUC: 0.566
hazelnut ROCAUC: 0.881
hazelnut pixel ROCAUC: 0.830
leather ROCAUC: 0.954
leather pixel ROCAUC: 0.615
metal_nut ROCAUC: 0.710
metal_nut pixel ROCAUC: 0.509
pill ROCAUC: 0.802
pill pixel ROCAUC: 0.647
screw ROCAUC: 0.667
screw pixel ROCAUC: 0.583
tile ROCAUC: 0.965
tile pixel ROCAUC: 0.632
toothbrush ROCAUC: 0.889
toothbrush pixel ROCAUC: 0.568
transistor ROCAUC: 0.903
transistor pixel ROCAUC: 0.507
wood ROCAUC: 0.959
wood pixel ROCAUC: 0.644
zipper ROCAUC: 0.966
zipper pixel ROCAUC: 0.415
Average ROCAUC: 0.854
Average pixel ROCUAC: 0.602

▼

[code after]

                # calculate distance matrix
                dist_matrix_list = []
                # feat_gallery = feat_gallery[:(feat_gallery.shape[0] // 100) * 100]
                print('feat_gallery.shape =', feat_gallery.shape)
                for d_idx in range(feat_gallery.shape[0] // 100 + 1):  # fixed, add one
                    # print('[d_idx*100:d_idx*10+100] = [%d:%d]' % (d_idx*100, d_idx*100+100))
                    dist_matrix = torch.pairwise_distance(feat_gallery[d_idx * 100:d_idx * 100 + 100], test_feat_map)
                    dist_matrix_list.append(dist_matrix)
                dist_matrix = torch.cat(dist_matrix_list, 0)
                print('dist_matrix.shape =', dist_matrix.shape)

[debug after]

feat_gallery.shape = torch.Size([15680, 256, 1, 1])
dist_matrix.shape = torch.Size([15680, 256, 56])
feat_gallery.shape = torch.Size([3920, 512, 1, 1])
dist_matrix.shape = torch.Size([3920, 512, 28])
feat_gallery.shape = torch.Size([980, 1024, 1, 1])
dist_matrix.shape = torch.Size([980, 1024, 14])

[measure after]

bottle ROCAUC: 0.972
bottle pixel ROCAUC: 0.562
cable ROCAUC: 0.848
cable pixel ROCAUC: 0.654
capsule ROCAUC: 0.897
capsule pixel ROCAUC: 0.638
carpet ROCAUC: 0.928
carpet pixel ROCAUC: 0.663
grid ROCAUC: 0.471
grid pixel ROCAUC: 0.566
hazelnut ROCAUC: 0.881
hazelnut pixel ROCAUC: 0.830
leather ROCAUC: 0.954
leather pixel ROCAUC: 0.615
metal_nut ROCAUC: 0.710
metal_nut pixel ROCAUC: 0.509
pill ROCAUC: 0.802
pill pixel ROCAUC: 0.647
screw ROCAUC: 0.667
screw pixel ROCAUC: 0.583
tile ROCAUC: 0.965
tile pixel ROCAUC: 0.632
toothbrush ROCAUC: 0.889
toothbrush pixel ROCAUC: 0.568
transistor ROCAUC: 0.903
transistor pixel ROCAUC: 0.507
wood ROCAUC: 0.959
wood pixel ROCAUC: 0.644
zipper ROCAUC: 0.966
zipper pixel ROCAUC: 0.415
Average ROCAUC: 0.854
Average pixel ROCUAC: 0.602

@byungjae89 Sorry to bother you again.

I then checked the operation and noticed that I could not reproduce the pixel ROCAUC you described. So, I followed the source code and found that it was due to an incorrect dimension for calculating the distance. Therefore, I have added the corrected version to this pull request. Please check it out.

[Your description vs This modified version]

[x] bottle : 97.0 vs 97.0
[x] cable : 92.3 vs 92.4
[x] capsule : 98.4 vs 98.4
[x] carpet : 98.9 vs 98.9
[x] grid : 98.3 vs 98.3
[x] hazelnut : 98.5 vs 98.5
[x] leather : 99.3 vs 99.3
[x] metal_nut : 97.1 vs 97.1
[x] pill : 95.0 vs 95.0
[x] screw : 99.1 vs 99.1
[x] tile : 92.8 vs 92.9
[x] toothbrush : 98.8 vs 98.8
[x] transistor : 86.6 vs 86.8
[x] wood : 95.3 vs 95.3
[x] zipper : 98.6 vs 98.6
[x] Average : 96.4 vs 96.4

The slight error may be due to the modification of the for-loop to expand its scope and allow the use of all layer functions, or it may be due to environmental differences.

Kind regards, :-D

byungjae89 / SPADE-pytorch

Extension of for-loop range and modification to reproduce pixel ROCAUC as described in README #18