An anomaly detection library comprising state-of-the-art algorithms and features such as experiment management, hyper-parameter optimization, and edge inference.
I would like to find appropriate hyper parameters using Comet. I run python tools/hpo/sweep.py --model padim --model_config custom_padim.yaml --sweep_config tools/hpo/configs/comet.yaml.
custom_padim.yaml:
name: tumar
format: folder
path: ../data
normal_dir: good # name of the folder containing normal images.
abnormal_dir: bad # name of the folder containing abnormal images.
normal_test_dir: null # name of the folder containing normal test images.
task: classification # classification or segmentation
mask: null #optional
extensions: null
split_ratio: 0.2 # ratio of the normal images that will be used to create a test split
image_size: 256
train_batch_size: 32
test_batch_size: 32
num_workers: 8
normalization: imagenet # data distribution to which the images will be normalized: [none, imagenet]
test_split_mode: from_dir # options: [from_dir, synthetic]
val_split_mode: same_as_test # options: [same_as_test, from_test, sythetic]
val_split_ratio: 0.5 # fraction of train/test images held out for validation (usage depends on val_split_mode)
transform_config:
train: null
val: null
create_validation_set: true
tiling:
apply: false
tile_size: null
stride: null
remove_border_count: 0
use_random_tiling: False
random_tile_count: 16
model:
name: padim
backbone: resnet18
pre_trained: true
layers:
- layer1
- layer2
- layer3
normalization_method: min_max # options: [none, min_max, cdf]
metrics:
image:
- F1Score
- AUROC
pixel:
- F1Score
- AUROC
threshold:
method: adaptive #options: [adaptive, manual]
manual_image: null
manual_pixel: null
visualization:
show_images: False # show images on the screen
save_images: False # save images to the file system
log_images: True # log images to the available loggers (if any)
image_save_path: null # path to which images will be saved
mode: full # options: ["full", "simple"]
project:
seed: 42
path: ./results
logging:
logger: [comet] # options: [comet, tensorboard, wandb, csv] or combinations.
log_graph: True # Logs the model graph to respective logger.
optimization:
export_mode: null # options: torch, onnx, openvino
# PL Trainer Args. Don't add extra parameter here.
trainer:
enable_checkpointing: true
default_root_dir: null
gradient_clip_val: 0
gradient_clip_algorithm: norm
num_nodes: 1
devices: [1]
enable_progress_bar: true
overfit_batches: 0.0
track_grad_norm: -1
check_val_every_n_epoch: 1 # Don't validate before extracting features.
fast_dev_run: false
accumulate_grad_batches: 1
max_epochs: 1
min_epochs: null
max_steps: -1
min_steps: null
max_time: null
limit_train_batches: 1.0
limit_val_batches: 1.0
limit_test_batches: 1.0
limit_predict_batches: 1.0
val_check_interval: 1.0 # Don't validate before extracting features.
log_every_n_steps: 50
accelerator: gpu # <"cpu", "gpu", "tpu", "ipu", "hpu", "auto">
strategy: null
sync_batchnorm: false
precision: 32
enable_model_summary: true
num_sanity_val_steps: 0
profiler: null
benchmark: false
deterministic: false
reload_dataloaders_every_n_epochs: 0
auto_lr_find: false
replace_sampler_ddp: true
detect_anomaly: false
auto_scale_batch_size: false
plugins: null
move_metrics_to_cpu: false
multiple_trainloader_mode: max_size_cycle
OpenVINO is not installed. Please install OpenVINO to use OpenVINOInferencer.
/home/r.abdrakhmanov/tumar/padim/anomalib/src/anomalib/config/config.py:280: UserWarning: config.project.unique_dir is set to False. This does not ensure that your results will be written in an empty directory and you may overwrite files.
warn(
Global seed set to 42
COMET INFO: 3f29a474fd714c8caf01957a0a9a47f4
COMET INFO: Using optimizer config: {'algorithm': 'bayes', 'configSpaceSize': 4, 'endTime': None, 'id': '3f29a474fd714c8caf01957a0a9a47f4', 'lastUpdateTime': None, 'maxCombo': 10, 'name': '3f29a474fd714c8caf01957a0a9a47f4', 'parameters': {'dataset.image_size': {'type': 'discrete', 'values': [128, 256]}, 'model.backbone': {'type': 'categorical', 'values': ['resnet18', 'wide_resnet50_2']}}, 'predictor': None, 'spec': {'gridSize': 10, 'maxCombo': 10, 'metric': 'image_F1Score', 'minSampleSize': 100, 'objective': 'maximize', 'retryAssignLimit': 0, 'retryLimit': 1000}, 'startTime': 23994265, 'state': {'mode': None, 'seed': None, 'sequence': [], 'sequence_i': 0, 'sequence_pid': None, 'sequence_retry': 0, 'sequence_retry_count': 0}, 'status': 'running', 'suggestion_count': 0, 'trials': 1, 'version': '2.0.23'}
COMET ERROR: Optimizer failed to retrieve data from the optimizer service, for more information look at: https://comet.com/docs/v2/api-and-sdk/python-sdk/warnings-errors/
Describe the solution you'd like
I would like to find appropriate hyper parameters using Comet
What is the motivation for this task?
I would like to find appropriate hyper parameters using Comet. I run
python tools/hpo/sweep.py --model padim --model_config custom_padim.yaml --sweep_config tools/hpo/configs/comet.yaml
. custom_padim.yaml:comet.iaml:
What I get:
Describe the solution you'd like
I would like to find appropriate hyper parameters using Comet
Additional context
No response