CUDA not working properly

[povolann]

Hello,

I edited the code since I need to run it for a whole data set, but when I try to run it with CUDA, it doesn't really work. After generating like 30 gradCAMs out of 235 images, I get this error:

free(): invalid pointer Aborted (core dumped)

root = '/home/anya/pytorch-grad-cam/CheXpert-v1.0-small/'
out_dir = './gradCAM/validation/'
data = []

# Index: -1 denotes multi-label mode including 14 diseases
DataSet =  CheXpert2(csv_path=root+'valid.csv',  image_root_path=root, use_upsampling=False, use_frontal=True, image_size=224, mode='train', class_index=-1)
loader =  torch.utils.data.DataLoader(DataSet, batch_size=32, num_workers=2, shuffle=False)

# model = models.resnet50(pretrained=True)
model = DenseNet121(last_activation=None, activations='relu', num_classes=14)
model.load_state_dict(torch.load('./DenseNet121_14classes_model.pth')) 
# set the evaluation mode - otherwise you can get very random results
model.eval()

And the second part...

for idx in DataSet.df.Path:
        # idx = 'valid/patient64589/study1/view1_frontal.jpg'
        patient = idx.split('/')[1] # patient number is 2nd item
        study = idx.split('/')[2]
        view = idx.split('/')[3].replace('.jpg', '')
        image_path = os.path.join(root, idx)
        rgb_img = cv2.imread(image_path, 1)[:, :, ::-1] # args.image_path

This is the loading part of the code for my data set. Thank you for any feedback :)

[jacobgil]

Hi, Can you please show how you call the model on the images?

[povolann]

Do you mean this part?

target_layers = [model.features[-2]] # not -1, that's batch normalisation

device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')

for idx, data in enumerate(loader):
        images, labels, imagepaths = data['image'], data['classes'], data['imagepath']
        images, labels  = images.to('cpu'), labels.to('cpu') # now I use cpu, since gpu didn't work
        y_pred = model(images) # torch.Size([32, 13])

[jacobgil]

Hi, I mean when you use the CAM object on the image batch and get the heatmap, Can you please show a snippet of how it's done there?

Some specifics I'm looking out to: is it re-creating the object it every time in the loop (it shouldn't)? Is it using the with statement?

[povolann]

Ok, so it should be this part of the code:

        rgb_img = cv2.imread(image_path, 1)[:, :, ::-1] # args.image_path
        rgb_img = np.float32(rgb_img) / 255
        input_tensor = preprocess_image(rgb_img,
                                        mean=[0.485, 0.456, 0.406],
                                        std=[0.229, 0.224, 0.225])

        # If None, returns the map for the highest-scoring category.
        # Otherwise, targets the requested category.
        target_category = None

        # Using the with statement ensures the context is freed, and you can
        # recreate different CAM objects in a loop.
        cam_algorithm = methods[args.method]
        with cam_algorithm(model=model,
                        target_layers=target_layers,
                        use_cuda=args.use_cuda) as cam:

            # AblationCAM and ScoreCAM have batched implementations.
            # You can override the internal batch size for faster computation.
            cam.batch_size = 32

            grayscale_cam = cam(input_tensor=input_tensor,
                                target_category=target_category,
                                aug_smooth=args.aug_smooth,
                                eigen_smooth=args.eigen_smooth)

            # Here grayscale_cam has only one image in the batch
            grayscale_cam = grayscale_cam[0, :]

            cam_image = show_cam_on_image(rgb_img, grayscale_cam, use_rgb=True)

            # cam_image is RGB encoded whereas "cv2.imwrite" requires BGR encoding.
            cam_image = cv2.cvtColor(cam_image, cv2.COLOR_RGB2BGR)

        gb_model = GuidedBackpropReLUModel(model=model, use_cuda=args.use_cuda)
        gb = gb_model(input_tensor, target_category=target_category)

        cam_mask = cv2.merge([grayscale_cam, grayscale_cam, grayscale_cam])
        cam_gb = deprocess_image(cam_mask * gb)
        gb = deprocess_image(gb)

Also, my full code can be found here: https://github.com/HYUNMIN-HWANG/AI604-Computer-Vision-Project/blob/main/pytorch-grad-cam/cam.py

[LCA-0907]

@povolann Hi, I'm using cam on my work and also ran into the problem that it only generates 1 cam image for each batch. Have you solved this problem?

I tried to get the cammask like cam_out = cam(input_tensor=x, targets=targets)[:,:] which will give the output map size like batch_size*w*h. However except for the first map , all of them are zeros.

[povolann]

@povolann Hi, I'm using cam on my work and also ran into the problem that it only generates 1 cam image for each batch. Have you solved this problem?

I tried to get the cammask like cam_out = cam(input_tensor=x, targets=targets)[:,:] which will give the output map size like batch_sizewh. However except for the first map , all of them are zeros.

Hi,

sorry for the late response, but I am not working anymore on this code, so you can see the last version on my Github page: https://github.com/HYUNMIN-HWANG/AI604-Computer-Vision-Project/blob/main/pytorch-grad-cam/cam.py

The code is quite messy though...