openvinotoolkit / anomalib

An anomaly detection library comprising state-of-the-art algorithms and features such as experiment management, hyper-parameter optimization, and edge inference.
https://anomalib.readthedocs.io/en/latest/
Apache License 2.0
3.41k stars 616 forks source link

fix winclip openvino inference output without mask #2083

Open junxnone opened 1 month ago

junxnone commented 1 month ago

๐Ÿ“ Description

WinClip OpenVINO Inference output is only the classification score.

โœจ Changes

Select what type of change your PR is:

โœ… Checklist

Before you submit your pull request, please make sure you have completed the following steps:

For more information about code review checklists, see the Code Review Checklist.

adrianboguszewski commented 1 month ago

@junxnone please check if it doesn't conflict with the order from DFM: https://github.com/openvinotoolkit/anomalib/pull/1952

junxnone commented 1 month ago
anomalib_winclip/train.py", line 21, in <module>
    export_torch = engine.export(model,ExportType.TORCH)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/anomalib_org/src/anomalib/engine/engine.py", line 937, in export
    exported_model_path = model.to_torch(
                          ^^^^^^^^^^^^^^^
  File "/anomalib_org/src/anomalib/models/components/base/export_mixin.py", line 83, in to_torch
    torch.save(
  File "/miniforge3/envs/tanomalib_org/lib/python3.11/site-packages/torch/serialization.py", line 628, in save
    _save(obj, opened_zipfile, pickle_module, pickle_protocol, _disable_byteorder_record)
  File "/miniforge3/envs/tanomalib_org/lib/python3.11/site-packages/torch/serialization.py", line 840, in _save
    pickler.dump(obj)
AttributeError: Can't pickle local object 'WinClipModel.encode_image.<locals>.get_feature_map.<locals>.hook'
from anomalib.data import MVTec
from anomalib.models import WinClip
from anomalib.engine import Engine
from anomalib.deploy import ExportType

datamodule = MVTec()
model = WinClip()
engine = Engine()

engine.fit(datamodule=datamodule, model=model)

export_torch = engine.export(model,ExportType.TORCH)
junxnone commented 1 month ago

root cause for this issue is here:

https://github.com/openvinotoolkit/anomalib/blob/e5b91d6a32c469ec0b9c4c161272b68114ba6fa3/src/anomalib/deploy/inferencers/openvino_inferencer.py#L150 https://github.com/openvinotoolkit/anomalib/blob/e5b91d6a32c469ec0b9c4c161272b68114ba6fa3/src/anomalib/deploy/inferencers/openvino_inferencer.py#L109

    11:22:44.293023 line       151         output_blob = compile_model.output(0)
    New var:....... output_blob = <ConstOutput: names[output] shape[?] type: f32>

https://github.com/openvinotoolkit/anomalib/blob/e5b91d6a32c469ec0b9c4c161272b68114ba6fa3/src/anomalib/deploy/inferencers/openvino_inferencer.py#L257-L268

    11:22:45.720814 line       258         predictions = predictions[self.output_blob]
    Modified var:.. predictions = array([0.5035419], dtype=float32)
    11:22:45.721448 line       261         anomaly_map: np.ndarray | None = None
    New var:....... anomaly_map = None
    11:22:45.721725 line       262         pred_label: LabelName | None = None
    New var:....... pred_label = None
    11:22:45.721992 line       263         pred_mask: float | None = None
    New var:....... pred_mask = None
    11:22:45.722259 line       267         if len(predictions.shape) == 1:
    11:22:45.722547 line       268             task = TaskType.CLASSIFICATION
    New var:....... task = <TaskType.CLASSIFICATION: 'classification'>
junxnone commented 1 month ago

Looks like the pred_score for segmentation task is not the correct way to get:

https://github.com/openvinotoolkit/anomalib/blob/e5b91d6a32c469ec0b9c4c161272b68114ba6fa3/src/anomalib/deploy/inferencers/openvino_inferencer.py#L266-L272