search

jayliu0313 / Shape-Guided

Shape-Guided Dual-Memory Learning for 3D Anomaly Detection [ICML2023]
35 stars 6 forks source link

The poor results #3

Closed LiZhi-CASIA closed 1 year ago

LiZhi-CASIA commented 1 year ago

I didn't train the 3D expert model myself. I directly used the best checkpoint of the 3D expert model in ./checkpoint. However, I obtained relatively poor results as shown in the figure below. I'm wondering if it might be because the checkpoint isn't very good? Also, if I want to train the 3D expert model to get the optimal parameters, what should I do? Could you provide more details? Thank you for the excellent work you've done. I'm looking forward to your response~

Shape_Guided result

jayliu0313 commented 1 year ago

I think there might be a configuration mistake. Could you give me more detail information or screenshot related to the settings? Additionally, please ensure that the data has undergone preprocessing and cut_patch.

LiZhi-CASIA commented 1 year ago

I think there might be a configuration mistake. Could you capture more pictures related to the settings? Additionally, please ensure that the data has undergone preprocessing and cut_patch.

Yes, the data has already undergone preprocessing and cut_patch. All the settings remained almost unchanged and I only fixed some minor bugs. For example, in the 'core/features.py': line 124 is added.

image

and line 223 in the original file is modified to line 225 in my file, and line 229 is also added:

image

Apart from the changes mentioned above, the other minor modifications are refinements of different versions of functions, which should not affect the results.

Below are two additional screenshots, one for alignment.txt and the other for Configuration.txt.

image image
jayliu0313 commented 1 year ago

This is a bit strange since we have tested different patches in the past, and the scores are quite similar. Additionally, we use a pretrained encoder for RGB, and it shouldn't result in such low scores. It might be necessary to spend some time to figure out this issue, and I will discuss with our team and reply you later. I appreciate your feedback regarding the bugs. We have made adjustments to address some minor issues.

jayliu0313 commented 1 year ago

Hello, we have retested everything, using the same code, variables, and checkpoints as on GitHub. However, the results we display still appear to be normal. Your alignment.txt and configuration.txt seem to be normal too. Please double-check whether all the paths are correct and the processes are being executed in the correct sequence. If you have any new leads, please kindly inform us. Here is our results retested on our github code. image

LiZhi-CASIA commented 1 year ago

The issue has been resolved. Thank you!

jayliu0313 commented 1 year ago

You're welcome! However, I would like to ask you why this issue has occurred?

LiZhi-CASIA commented 1 year ago

You're welcome! However, I would like to ask you why this issue has occurred?

It sounds a bit foolish, because I only preprocessed the training dataset.

LiZhi-CASIA commented 1 year ago

I have one more question. Is pretraining the 3D expert model required using datasets other than Mvtec3D？

jayliu0313 commented 1 year ago

Thanks for your feedback! No, we just use MVTec3D-AD dataset to train our 3D expert.

LiZhi-CASIA commented 1 year ago

Thanks for your feedback! No, we just use MVTec3D-AD dataset to train our 3D expert.

Thanks for your timely reply. But when I modify the variable 'IS_PRTRAIN = True' and rerun the 'cut_patches.py', it is unable to generate the npz file. image It seems to be because there is no 'pretrain' folder under the mvtec3D dataset. Could you provide more details?

jayliu0313 commented 1 year ago

I'm sorry. We haven't done a good job in terms of customization in this aspect. In the future, we will rewrite the "cut_patches.py" to provide pre-trained data. Alternatively, if you're in a hurry to need, our approach is to randomly select 25 samples of tiff files from each category of the preprocessed training data as our pretrained data, and save them in the pretrained data folder. Next, you can use this pretrained data folder as your dataset_path. If our work finished, we will tell you.

LiZhi-CASIA commented 1 year ago

Thanks again！I will try to obtain the necessary npz files for training as you mentioned. Meanwhile, I'm looking forward to your future work. I'll keep following it.

jayliu0313 commented 1 year ago

Hi, we have already rewritten the "README" and related code. You can follow the instructions to complete the "train 3D expert model". Additionally, I suggest that you can re-clone our code. If there is any question, please tell me. Thank you!!