I am trying the model on Yolov5 but it's giving an error every time:

[akshay-gupta123]

❔I am trying the model on LIME but it's giving an error every time:

error                                     Traceback (most recent call last)
<ipython-input-337-fb130e3f2aaf> in <module>()
      1 
      2 # Hide color is the color for a superpixel turned OFF. Alternatively, if it is NONE, the superpixel will be replaced by the average of its pixels
----> 3 explanation = explainer.explain_instance(np.array(img),batch_predict, top_labels=5, hide_color=0, num_samples=1000)

6 frames
/root/.cache/torch/hub/ultralytics_yolov5_master/utils/datasets.py in letterbox(img, new_shape, color, auto, scaleFill, scaleup)
    804 
    805     if shape[::-1] != new_unpad:  # resize
--> 806         img = cv2.resize(img, new_unpad, interpolation=cv2.INTER_LINEAR)
    807     top, bottom = int(round(dh - 0.1)), int(round(dh + 0.1))
    808     left, right = int(round(dw - 0.1)), int(round(dw + 0.1))

error: OpenCV(4.1.2) /io/opencv/modules/imgproc/src/resize.cpp:3363: error: (-215:Assertion failed) !dsize.empty() in function 'resize'

Here is my code:

model = torch.hub.load('ultralytics/yolov5','custom', path_or_model='/content/best.pt')
model = model.autoshape() 

def get_pil_transform(): 
    transf = transforms.Compose([
        transforms.Resize((256, 256)),
        transforms.CenterCrop(224)
    ])    

    return transf

def get_preprocess_transform():
    normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],
                                    std=[0.229, 0.224, 0.225])     
    transf = transforms.Compose([
        transforms.ToTensor(),
        normalize
    ])    

    return transf    

pill_transf = get_pil_transform()
preprocess_transform = get_preprocess_transform()

def batch_predict(images):
    batch = torch.stack(tuple(preprocess_transform(i) for i in images), dim=0)
    logits = model(images)
    result = numpy.array(logits.xyxy[0][:,4])
    result = [[i,1-i,0] for i in result]
    return result
def get_image(path):
    with open(os.path.abspath(path), 'rb') as f:
        with Image.open(f) as img:
            return img.convert('RGB') 

img = get_image('/content/image-mask-estimate/data/sample-images/2.jpg')

explainer = lime_image.LimeImageExplainer()
explanation = explainer.explain_instance(np.array(pill_transf(img)),batch_predict , top_labels=5, hide_color=0, num_samples=100)

Additional context

It's working fine on Yolov5

test_pred = batch_predict([pill_transf(img)])
print(test_pred)

[chaitanyabunny]

Hi @akshay-gupta123 , I am also facing same problem. I am using LIME explainer for yolov5 model and after running the explainer, getting same error as you have mentioned. Please post the solution, how did you resolved this error ? It would be really helpful.

[chaitanyabunny]

Here is my code,

code_repository_path = '/root/AMP-SSV-Reference/code/s2/model/yolov5'

model = torch.hub.load(code_repository_path,'custom', path='/root/AMP-SSV-Reference/code/s2/model/yolov5/k2_best.pt', source='local',force_reload=True)

model = model.autoshape()

def get_pil_transform(): transf = transforms.Compose([ transforms.Resize((256, 256)), transforms.CenterCrop(224) ])

return transf

def get_preprocess_transform(): normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
transf = transforms.Compose([ transforms.ToTensor(), normalize ])

return transf    

pill_transf = get_pil_transform() preprocess_transform = get_preprocess_transform()

def batch_predict(images): batch = torch.stack(tuple(preprocess_transform(i) for i in images), dim=0) logits = model(images) result = numpy.array(logits.xyxy[0][:,4]) result = [[i,1-i,0] for i in result] return result def get_image(path): with open(os.path.abspath(path), 'rb') as f: with Image.open(f) as img: return img.convert('RGB')

img = get_image('/root/5.jpg') plt.imshow(img)

explainer = lime_image.LimeImageExplainer() explanation = explainer.explain_instance(np.array(pill_transf(img)),batch_predict , top_labels=5, hide_color=0, num_samples=100)

Following is the error:

~/AMP-SSV-Reference/code/s2/model/yolov5/models/common.py in forward(self, imgs, size, augment, profile) 593 imgs[i] = im if im.data.contiguous else np.ascontiguousarray(im) # update 594 shape1 = [make_divisible(x, self.stride) if self.pt else size for x in np.array(shape1).max(0)] # inf shape --> 595 x = [letterbox(im, shape1, auto=False)[0] for im in imgs] # pad 596 x = np.ascontiguousarray(np.array(x).transpose((0, 3, 1, 2))) # stack and BHWC to BCHW 597 x = torch.from_numpy(x).to(p.device).type_as(p) / 255 # uint8 to fp16/32

~/AMP-SSV-Reference/code/s2/model/yolov5/models/common.py in (.0) 593 imgs[i] = im if im.data.contiguous else np.ascontiguousarray(im) # update 594 shape1 = [make_divisible(x, self.stride) if self.pt else size for x in np.array(shape1).max(0)] # inf shape --> 595 x = [letterbox(im, shape1, auto=False)[0] for im in imgs] # pad 596 x = np.ascontiguousarray(np.array(x).transpose((0, 3, 1, 2))) # stack and BHWC to BCHW 597 x = torch.from_numpy(x).to(p.device).type_as(p) / 255 # uint8 to fp16/32

~/AMP-SSV-Reference/code/s2/model/yolov5/utils/augmentations.py in letterbox(im, new_shape, color, auto, scaleFill, scaleup, stride) 115 116 if shape[::-1] != new_unpad: # resize --> 117 im = cv2.resize(im, new_unpad, interpolation=cv2.INTER_LINEAR) 118 top, bottom = int(round(dh - 0.1)), int(round(dh + 0.1)) 119 left, right = int(round(dw - 0.1)), int(round(dw + 0.1))

error: OpenCV(4.6.0) /io/opencv/modules/imgproc/src/resize.cpp:3689: error: (-215:Assertion failed) !dsize.empty() in function 'resize'

[akshay-gupta123]

Hey @chaitanyabunny, Lime isn't compatible with the detector output format. For each perturbation, you might get different-sized output vectors, which LIME wouldn't be able to handle as it is primarily designed for classification tasks which has fixed sized output vector. So, you have to somehow convert your dynamic sized vector into a fixed size vector. Logic depends on your task. Hope this helps.

[chaitanyabunny]

Hi @akshay-gupta123 ..Thanks for the reply. Yes, and what you said, making sense to me. LIME is basically meant for classification, segmentation tasks, and current api of LIME doesn't support detections task.. I understood the root cause.. Thanks @akshay-gupta123