training patchcore

[aytek03]

Hi sir

I want to train this network but I cannot.

I write this code in terminal:

datapath=/path/to/data/from/mvtec

I got : datapath: command not found

How can I write on terminal?

[aytek03]

Also When I write this:

python bin/run_patchcore.py --gpu 0 --seed 0 --save_patchcore_model --log_group IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0 --log_project MVTecAD_Results results \ patch_core -b wideresnet50 -le layer2 -le layer3 --faiss_on_gpu --pretrain_embed_dimension 1024 --target_embed_dimension 1024 --anomaly_scorer_num_nn 1 --patchsize 3 sampler -p 0.1 approx_greedy_coreset dataset --resize 256 --imagesize 224 "${dataset_flags[@]}" mvtec $datapath

I got this:

Usage: run_patchcore.py [OPTIONS] RESULTS_PATH COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...

Options: --gpu INTEGER [default: 0] --seed INTEGER [default: 0] --log_group TEXT --log_project TEXT --save_segmentation_images --save_patchcore_model --help Show this message and exit.

Commands: dataset patch_core sampler

[cyj95]

have u solved this problem?

[aytek03]

No I have not yet. @cyj95

[ez615]

Did you put enter between '\' and 'patch_core'? I think that's the only difference from the code I wrote.

[petergehler]

I would bet this is due to the extraneous \ in your command so click does not see the patch_core command. In one line this would be the call I would try:

python bin/run_patchcore.py --gpu 0 --seed 0 --save_patchcore_model --log_group IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0 --log_project MVTecAD_Results results patch_core -b wideresnet50 -le layer2 -le layer3 --faiss_on_gpu --pretrain_embed_dimension 1024 --target_embed_dimension 1024 --anomaly_scorer_num_nn 1 --patchsize 3 sampler -p 0.1 approx_greedy_coreset dataset --resize 256 --imagesize 224 "${dataset_flags[@]}" mvtec $datapath```

[aytek03]

@petergehler That's right. I'll use it.

However, I still cannot get these lines:

datapath=/path_to_mvtec_folder/mvtec datasets=('bottle' 'cable' 'capsule' 'carpet' 'grid' 'hazelnut' 'leather' 'metal_nut' 'pill' 'screw' 'tile' 'toothbrush' 'transistor' 'wood' 'zipper') dataset_flags=($(for dataset in "${datasets[@]}"; do echo '-d '$dataset; done))

I wrote these in terminal but as you can guess

bash: datapath: command not found.

How can I define datapath and dataset in terminal?

[aytek03]

I solve this problem. And this is the script.

python3 bin/run_patchcore.py --gpu 0 --seed 0 --save_patchcore_model --log_group IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0 --log_project MVTecAD_Results results patch_core -b wideresnet50 -le layer2 -le layer3 --faiss_on_gpu --pretrain_embed_dimension 1024 --target_embed_dimension 1024 --anomaly_scorer_num_nn 1 --patchsize 3 sampler -p 0.1 approx_greedy_coreset dataset --resize 256 --imagesize 224 "${dataset_flags[@]}" mvtec $"/home/ec2-user/mvtec"

I change the last part. I write mvtec $"/home/ec2-user/mvtec" instead of mvtec $datapath.

Our dataset folder is /home/ec2-user/mvtec

[kk206]

@aytek03 Thank you for opening this issue.

How did you solve these lines: datapath=/path_to_mvtec_folder/mvtec datasets=('bottle' 'cable' 'capsule' 'carpet' 'grid' 'hazelnut' 'leather' 'metal_nut' 'pill' 'screw' 'tile' 'toothbrush' 'transistor' 'wood' 'zipper') dataset_flags=($(for dataset in "${datasets[@]}"; do echo '-d '$dataset; done))

I don't know how can I define datasets and dataset_flags.

[aytek03]

Hi @kk206

I discard these there lines:

datapath=/path_to_mvtec_folder/mvtec datasets=('bottle' 'cable' 'capsule' 'carpet' 'grid' 'hazelnut' 'leather' 'metal_nut' 'pill' 'screw' 'tile' 'toothbrush' 'transistor' 'wood' 'zipper') dataset_flags=($(for dataset in "${datasets[@]}"; do echo '-d '$dataset; done))

and I write this script and start to train the model:

python bin/run_patchcore.py --gpu 0 --seed 0 --save_patchcore_model --log_group IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0 --log_project MVTecAD_Results results patch_core -b wideresnet50-le layer2 -le layer3 --pretrain_embed_dimension 1024 --target_embed_dimension 1024 --anomaly_scorer_num_nn 1 --patchsize 3 sampler -p 0.1 approx_greedy_coreset dataset --resize 256 --imagesize 224 --subdatasets "bottle" mvtec "your/mvtec/datapath"

I also add --subdatasets "bottle" then my model trains the bottle. (discard --faiss_on_gpu)

[kk206]

@aytek03 Thank you for your reply.

I could train my model. I didn't need to discard --faiss_on_gpu on my PC.

Thank you.

[tuandat-github]

Hello @aytek03, your code is only run on "bottle". How about the others?

[aytek03]

Hello @tuandat-github ,

You implement the code class by class. You can change this line "bottle" to such as "carpet". I have not tried all classes at the same time for training.

[tuandat-github]

@aytek03 Do you know any code for running all classes at the same time?

[aytek03]

@tuandat-github I do not know.

[tuandat-github]

@aytek03 Can you give me a code for evaluating? Thank you.

[aytek03]

@tuandat-github

python bin/load_and_evaluate_patchcore.py --gpu 0 --seed 0 &savefolder patch_core_loader -p "your/pkl data/Patchcore/results/MVTecAD_Results/IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0_5/models/mvtec_bottle/patchcore_params.pkl" dataset --resize 366 --imagesize 320 --subdatasets "bottle" mvtec "path/your/mtvec/data"

[1BTU]

Do you know why I'm reporting this error here? ModuleNotFoundError: No module named 'patchcore' ,I'm sure I have patchcore in my file and the path is "src/patchcore"

[chenhao324]

@1BTU you could try to change the "import patchcore...." into "import src.patchcore", and so on

[Leonardo0325]

@1BTU This is a command line problem，I also encountered this problem.Have you solved the problem?

[yuhbai]

@tuandat-github

python bin/load_and_evaluate_patchcore.py --gpu 0 --seed 0 &savefolder patch_core_loader -p "your/pkl data/Patchcore/results/MVTecAD_Results/IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0_5/models/mvtec_bottle/patchcore_params.pkl" dataset --resize 366 --imagesize 320 --subdatasets "bottle" mvtec "path/your/mtvec/data"

how to debug this code

[romanvelichkin]

Correct way to evaluate model with one dataset, ex. "bottle" (with faiss-gpu):

python bin/load_and_evaluate_patchcore.py --gpu 0 --seed 0 "folder_to_save_result" patch_core_loader -p "models/IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1/models/mvtec_bottle/" dataset --resize 256 --imagesize 224 -d "bottle" mvtec "folder_with_mvtec_datasets"

python bin/load_and_evaluate_patchcore.py we run patchcore

--gpu 0 --seed 0 set number of gpu you want to use and seed (result will be same with same seed)

"folder_to_save_result" path to folder where you want to save results

patch_core_loader -p "models/IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1/models/mvtec_bottle/" path to folder with weights you want to use

dataset --resize 256 --imagesize 224 images will be resized to 256 and then cropped to 224, final size must be appropriate for used model

-d "bottle" mvtec "folder_with_mvtec_datasets" dataset you want to use, with path to parent folder with all mvtec datasets

[romanvelichkin]

Example with evaluating multiple datasets (with faiss-gpu):

python bin/load_and_evaluate_patchcore.py --gpu 0 --seed 0 "save_folder" patch_core_loader -p "models/IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1/models/mvtec_bottle/" -p "models/IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1/models/mvtec_cable/" dataset --resize 256 --imagesize 224 -d "bottle" -d "cable" mvtec "folder_with_mvtec_datasets"

[eklahari]

I am getting an error as no module named patchcore while executing run_patchcore.py

[Swaggyyoung03]

Do you know why I'm reporting this error here? ModuleNotFoundError: No module named 'patchcore' ,I'm sure I have patchcore in my file and the path is "src/patchcore"

try: python setup.py install

[cugwu]

Dear all I still facing a similar issue when trying to run the code. I've already tried using python3 instead of python, I tried using directly the data folder in the arergumets of "dataset" meaning using $"/data/users/cugwu/ad_data/mvtec" instead of $datapath, I tried to remove "\" and write all the command in the same line but nothing is working. I also tried to put directly on run_patchcore.py as default values the parameters of "dataset", "sampler" and "patch_core" but I still get the same error.

This is what I'm writing:

datapath=/data/users/cugwu/ad_data/mvtec
results_path=/data/users/cugwu/ad_data/results/F4AD/patchcore/IN/mvtec
datasets=('bottle'  'cable'  'capsule'  'carpet'  'grid'  'hazelnut' 'leather'  'metal_nut'  'pill' 'screw' 'tile' 'toothbrush' 'transistor' 'wood' 'zipper')
dataset_flags=($(for dataset in "${datasets[@]}"; do echo '-d '$dataset; done))

export PYTHONPATH=src

python bin/run_patchcore.py --gpu 0 --seed 0 --save_patchcore_model \
--log_group IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0 --log_project MVTecAD_Results results $results_path \
patch_core -b wideresnet50 -le layer2 -le layer3 --faiss_on_gpu \
--pretrain_embed_dimension 1024  --target_embed_dimension 1024 --anomaly_scorer_num_nn 1 --patchsize 3 \
sampler -p 0.1 approx_greedy_coreset dataset --resize 256 --imagesize 224 "${dataset_flags[@]}" mvtec $datapath

and this is what I keep getting:

INFO:__main__:Command line arguments: bin/run_patchcore.py --gpu 0 --seed 0 --save_patchcore_model --log_group IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0 --log_project MVTecAD_Results results /data/users/cugwu/ad_data/results/F4AD/patchcore/IN/mvtec patch_core -b wideresnet50 -le layer2 -le layer3 --faiss_on_gpu --pretrain_embed_dimension 1024 --target_embed_dimension 1024 --anomaly_scorer_num_nn 1 --patchsize 3 sampler -p 0.1 approx_greedy_coreset dataset --resize 256 --imagesize 224 -d bottle -d cable -d capsule -d carpet -d grid -d hazelnut -d leather -d metal_nut -d pill -d screw -d tile -d toothbrush -d transistor -d wood -d zipper mvtec /data/users/cugwu/ad_data/mvtec

Usage: run_patchcore.py [OPTIONS] RESULTS_PATH COMMAND1 [ARGS]... [COMMAND2
                        [ARGS]...]...

Options:
  --gpu INTEGER               [default: 0]
  --seed INTEGER              [default: 0]
  --log_group TEXT
  --log_project TEXT
  --save_segmentation_images
  --save_patchcore_model
  --help                      Show this message and exit.

Commands:
  dataset
  patch_core
  sampler

Could someone please assist me or offer any suggestions on how to resolve this issue?

[petergehler]

@cugwu The problem in your call is that the first argument that run_patchcore expects is a RESULTS_PATH. That should come after the [OPTIONS] and before the COMMAND1. At that point in your command line you write results $result_path and results is interpreted as the RESULTS_PATH. So if you change results $results_path to $results_path it should work.

python bin/run_patchcore.py --gpu 0 --seed 0 --save_patchcore_model \
--log_group IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0 --log_project MVTecAD_Results $results_path \
patch_core -b wideresnet50 -le layer2 -le layer3 --faiss_on_gpu \
--pretrain_embed_dimension 1024  --target_embed_dimension 1024 --anomaly_scorer_num_nn 1 --patchsize 3 \
sampler -p 0.1 approx_greedy_coreset dataset --resize 256 --imagesize 224 "${dataset_flags[@]}" mvtec $datapath