I don't have ground_truth image,how can I modify it before I can work it

[tiskw]

Hi @DeepSpace98, thank you for your interest in my repository!

Let me confirm, you wanted to train a model using your own dataset that doesn't have ground truth images (mask images), correct?

If yes, you don't need to modify the code. The main.py can handle the custom dataset even if the dataset doesn't have ground truth images. See the "Train and predict on your dataset" section in the README.md for more details.

In short, please place your training data (only good images) and test data under some directory (e.g. data_train/ and data_test/ respectively), then run the following commands. The detection results are stored under output_test/ diretory.

# Training
python3 main.py train -i data_train -o ./index.faiss

# Test
python3 main.py predict -i data_test -o output_test

[DeepSpace98]

Hi @DeepSpace98, thank you for your interest in my repository!

Let me confirm, you wanted to train a model using your own dataset that doesn't have ground truth images (mask images), correct?

If yes, you don't need to modify the code. The main.py can handle the custom dataset even if the dataset doesn't have ground truth images. See the "Train and predict on your dataset" section in the README.md for more details.

In short, please place your training data (only good images) and test data under some directory (e.g. data_train/ and data_test/ respectively), then run the following commands. The detection results are stored under output_test/ diretory.

# Training
python3 main.py train -i data_train -o ./index.faiss

# Test
python3 main.py predict -i data_test -o output_test

My dateset's image shape is (500,400),but this project is (224,224) when I want to change the image shape,the train is working,but when I work to predict, anomaly_score =score_patches.shape((28,28,-1)) report errors.

[tiskw]

Hi @DeepSpace98,

I see, it make sense to me. I guess you used --input_size option or manually changed the input image size in the code. I will update my implementation within a few days to be able to treat input images with arbitrarily shape, and report here.

If you want a quick bug fix, please see the following suggestion. However, I not sure the suggestion works well on your code, because, I guess, you've already changed my code a lot (my current code should not be able to treat non-square shape image, just forcibly resize all input images to 224x224 by default).

As you pointed out, the error was caused by the magic number (28, 28, -1) which is the shape of NN features when the input shape is (3, 224, 224). The error location probably is this line. For quickly solving this issue, you can (1) check the feature shape, and (2) replace (28, 28, -1) to the appropriate number.

(1) Check the feature shape Please insert the following debug print code in patchcore/extractor.py L164-165, and run the test command again.

embedding = self.forward(data)
print(embedding.shape)   # Intert this code
return flatten_NHW(embedding).cpu().numpy()

The above debug code prints (1, C, H, W), so please memorize H and W. After then, you can remove the above debug code.

(2) Replace (28, 28, -1) to the appropriate number. Please replace score_patches.reshape((28, 28, -1)) in patchcore/patchcore.py L179 to score_patches.reshape((H, W, -1)) where H and W are the numbers you've got in (1).

If the above measure does not work, please wait my update.

[DeepSpace98]

Hi @DeepSpace98,

I see, it make sense to me. I guess you used --input_size option or manually changed the input image size in the code. I will update my implementation within a few days to be able to treat input images with arbitrarily shape, and report here.

If you want a quick bug fix, please see the following suggestion. However, I not sure the suggestion works well on your code, because, I guess, you've already changed my code a lot (my current code should not be able to treat non-square shape image, just forcibly resize all input images to 224x224 by default).

As you pointed out, the error was caused by the magic number (28, 28, -1) which is the shape of NN features when the input shape is (3, 224, 224). The error location probably is this line. For quickly solving this issue, you can (1) check the feature shape, and (2) replace (28, 28, -1) to the appropriate number.

(1) Check the feature shape Please insert the following debug print code in patchcore/extractor.py L164-165, and run the test command again.

embedding = self.forward(data)
print(embedding.shape)   # Intert this code
return flatten_NHW(embedding).cpu().numpy()

The above debug code prints (1, C, H, W), so please memorize H and W. After then, you can remove the above debug code.

(2) Replace (28, 28, -1) to the appropriate number. Please replace score_patches.reshape((28, 28, -1)) in patchcore/patchcore.py L179 to score_patches.reshape((H, W, -1)) where H and W are the numbers you've got in (1).

If the above measure does not work, please wait my update.

Thanks，it helps me very much.And now I have some new question.When I want to test my image ,I find `def getitem(self, idx): """ Returns idx-th data. Args: idx (int): Index of image to be returned. """

Load input image.

    path = self.paths[idx]
    img  = PIL.Image.open(str(path)).convert("RGB")
    # Apply transforms.
    img = self.transform(img)
    # Returns only image while keeping the same interface as the MVTecAD class.
    return (img, 0, 0, str(path), 0)` use MVTecADImageOnly to load dateset ,it can't back the image labels,so I can't use Patchcore.score() to get auc-roc score.What should I do to achieve this？hope you can help me.

[tiskw]

Hi @DeepSpace98,

Sounds good! I'm happy to hear that.

OK I understood, you have image labels (but don't have ground-truth mask images) and wanted to compute the image-level auc-roc score, correct?

If my above understanding is correct, I suggest you to modify the MVTecADImageOnly.__getitem__ (or, define a new class). I think you need 2 steps, (1) add code for getting image label information, and (2) return the label and dummy ground-truth image.

(1) add code for getting image-wise label information Please add code for getting the label information in patchcore/dataset.py, L 221, and store the information in a variable flag_an. The flag_an should be 0 or 1 where 0 means good image and 1 means failure. I think the following variables are helpful for getting the label information.

• path: path to the image file (this is not str, but pathlib.Path instance)
• self.root: root directory of the train/test images (this is also pathlib.Path instance too)

The following is an example in the case of MVTecAD dataset which has the label information in the sub-directory name:

flag_an = 0 if (path.parent.name == "good") else 1

(2) return the label and dummy ground truth image Please replace the return statement of the MVTecADImageOnly.__getitem__ to the following code:

# Generate dummy ground-truth image at random.
gt = (torch.rand((img.shape[1:])) + 0.5).int().float()

# Returns input image with label and dummy ground-truth image
return (img, gt, flag_an, str(path), 0)

The gt is necessary for the above code, because PatchCore.score will try to compute both image-level and pixel-level scores and it will raise an error if the ground-truth image is empty.

Now, you can call PatchCore.score. Note that the pixel-level auc-roc score is meaningless. I guess the pixel-level score should be around 0.5 because the gt is a uniform sampling from 0 and 1 (0 means good and 1 means failure).

If you have any other questions, feel free to keep this thread!

P.S. If you have both image labels and ground-truth mask images, you can use MVTecAD class instead of MVTecADImageOnly. In this case, you need to place your custom dataset in the same directory structure as MVTecAD dataset.

[DeepSpace98]

Hi @tiskw , when I try to predict my datesets image ,I find a question, in patchcore.py L106 anomaly_map_rw, score = self.compute_anomaly_scores(score_patches, x.shape), the 'score' is range(1,10),but how can I decision the threshold to distinguish the image is good or bad?

This is my data_test 's list_true_im_lvl and list_pred_im_lvl, [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] [2.1778686121091115, 2.043035511210147, 2.204647339770545, 1.8679341253990513, 1.8125410547287593, 1.9134133413679484, 2.018275448728007, 1.9188715219069212, 1.833431549788796, 1.8692872363283126, 1.859954622115166, 1.9081311584256666, 2.0998600457211163, 1.9449997 356788453, 1.8968703680027401, 1.8406372425886528, 1.8875199038513937, 1.9256044650201363, 2.1198249066929193, 2.6310014671811315, 2.3126015866927063, 2.600668857367263, 2.9402873803243894, 2.0643573969937936, 3.7993765604691614, 3.273141894090825, 3.51990609100607 2, 7.870590446232199, 9.73244190839107, 7.205947214190297, 9.043212991807236, 8.846725351483489, 11.231980665213301, 6.442720413923064, 7.837783801879729, 7.7509763145267305, 3.7662560338415814, 8.148498547063122, 6.9112773307720134, 10.825699244757274, 7.168914369 207783, 10.657363077996497, 9.78399165118162, 8.693112685231444, 7.544686819496402, 6.335824087741453, 5.186816725496845, 3.9724798250279982, 11.732951833917477, 10.984492020644685, 5.018968999991873]

if you ,how do you determine this threshold?

[tiskw]

Hi @DeepSpace98,

Yes, as you pointed out, you need the other technique to determine the threshold because PatchCore just provide anomaly scores which has high correlation with the true good/failure label. The threshold should be selected by the requirement for the anomaly detection algorithm like FPR (false positive rate = overdetection) or FNR (false negative rate = overlooking).

At first, Ideally, we need failure images for the threshold selection. If we have that, we can draw a histogram of anomaly scores for each good and failure images. The following figure is an example of the histogram in your test dataset. In this case 3.0~3.2 probably a good threshold.

This is a lucky case because there are no overlaps between good and failure histogram curve, that is, you can find a perfect threshold. However, sometimes we cannot find a perfect threshold. If so, ROC curve also provides a good information because ROC curve is a graph of FPR and TPR that are intuitive from application viewpoint.

If you don't have failure images, it is quite tough to select an appropriate threshold. However, I think the histogram of good image scores still helpful in this case.

Does it answer your question...?

[DeepSpace98]

Hi @DeepSpace98,

Yes, as you pointed out, you need the other technique to determine the threshold because PatchCore just provide anomaly scores which has high correlation with the true good/failure label. The threshold should be selected by the requirement for the anomaly detection algorithm like FPR (false positive rate = overdetection) or FNR (false negative rate = overlooking).

At first, Ideally, we need failure images for the threshold selection. If we have that, we can draw a histogram of anomaly scores for each good and failure images. The following figure is an example of the histogram in your test dataset. In this case 3.0~3.2 probably a good threshold.

This is a lucky case because there are no overlaps between good and failure histogram curve, that is, you can find a perfect threshold. However, sometimes we cannot find a perfect threshold. If so, ROC curve also provides a good information because ROC curve is a graph of FPR and TPR that are intuitive from application viewpoint.

If you don't have failure images, it is quite tough to select an appropriate threshold. However, I think the histogram of good image scores still helpful in this case.

Does it answer your question...?

Yes，This is very helpful to me. Thanks very much。

[tiskw]

Hi @DeepSpace98,

No problem, I'm glad I was able to support you! :+1:

I will close this issue because the contents of the question seems to be getting away a little from the title (and original question). However, please feel free to create a new issue if you have another question.