How to generate the density map from the output of model

[neverland0621]

Thank you for releasing the code. I have some questions about the source code. 1.How to generate the density map from the output of model?I didn't get the desity map from the output of the model when i run the val.ipython but only the groudtruth of density map.or ,how to transform the output of the model to densoty map?

2. I found that the sum of the density map from groundtruth is not same as the groundtruth count number of shanghaitech dataset in .mat file.even worse ,they are very different just as the picture.if the sum of groundtruth is far from the real count number ,what does the meaning of calculating the error of it. number output groundtruth 1 218.73355 265.32138 2 242.14491 232.52945 3 113.73707 65.99532 4 371.46692 372.23004 5 432.0569 306.81796 6 144.08446 190.22192 7 269.57242 269.53113 8 565.20056 565.5759 9 235.49048 307.76044 10 507.46124 510.92313 11 297.7716 298.135 12 317.20828 493.84286 13 665.5448 580.9203 14 536.8564 468.91245 15 189.81647 241.52519 16 558.702 583.2348 17 102.62774 95.6099 18 401.4803 376.6646 19 244.39113 211.23973 20 343.18445 373.51428 we can find that the difference of two count numbers from one picture.they are the count number from .mat file (.mat image_info number)and the count number from sum of generated density map.we can get the count number from sum of generated density map of the fist image in shanghaitech A from the data above ,it equals 265.32, however,the count number from .mat file of this image is 172. And not only this image,most of the data have the same problem. here is my code: mae = 0 for i in xrange(len(img_paths)): img = 255.0 * F.to_tensor(Image.open(img_paths[i]).convert('RGB')) img[0,:,:]=img[0,:,:]-92.8207477031 img[1,:,:]=img[1,:,:]-95.2757037428 img[2,:,:]=img[2,:,:]-104.877445883 img = img.cuda() img = transform(Image.open(img_paths[i]).convert('RGB')).cuda() gt_file = h5py.File(img_paths[i].replace('.jpg','.h5').replace('images','ground_truth'),'r') groundtruth = np.asarray(gt_file['density']) output = model(img.unsqueeze(0)) print i+1,output.detach().cpu().sum().numpy(),np.sum(groundtruth)

   print mse

   I will appreciate the help if anyone has idea of the problem, thank you!

[aditya-malte]

Hello, The output of the model can be converted to a heatmap by using matplotlib like this:

from matplotlib import cm plt.imshow(output , cmap = cm.jet )

This maps the raw [0,1] to H->[240, 0]. (H being hue from HSV).

PS. For your second doubt: The count values might vary a tiny bit due to the gaussian blur(this is because only an infinitely large kernel can approximate the gaussian blur accurately)

Thanks

[neverland0621]

@aditya-malte Thank you for your help! The problems have been solved.^_^

[wait1988]

Can you tell me how to solve it?

[aditya-malte]

@neverland0621 you're welcome

[aditya-malte]

@wait1988 you could use the code I have shown in my previous comment

[SJLNNU]

@aditya-malte Could you tell me more about the heatmap . I use firest test imagein shanghai tech dataset .The output (output.detach().cpu().numpy())'s size is 11264,but the input image size is 1024*704 ,I can't get a heatmap for each pix in the input image. Plz help me out.

[aditya-malte]

Hi SJLNNU, I didn't understand what you mean to say exactly. Check my answers to say if they help. Regards, Aditya Malte

[SJLNNU]

@aditya-malte Hi ,sorry for my statement . There is the thing : I Add "plt.imshow(output.detach().cpu().numpy() , cmap = cm.jet )” at the end of code just like you said . But it returns

---

Traceback (most recent call last): File "G:/CrowdDetection/CSRNet-pytorch-master/val.py", line 64, in plt.imshow(output.detach().cpu().numpy() , cmap = cm.jet ) File "C:\Users\ASUS\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\pyplot.py", line 3205, in imshow *kwargs) File "C:\Users\ASUS\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib__init__.py", line 1855, in inner return func(ax, args, **kwargs) File "C:\Users\ASUS\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\axes_axes.py", line 5487, in imshow im.set_data(X) File "C:\Users\ASUS\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\image.py", line 653, in set_data raise TypeError("Invalid dimensions for image data") TypeError: Invalid dimensions for image data

---

The Size of 'output.detach().cpu().numpy()' is not enough to reshape into input image size. What should I do to out put the heat Image? Thank you for your time .

[aditya-malte]

Hello SJLNNU, The problem appears to be from some other part of your code then, because I haven't faced such an issue.

Best, Aditya Malte

[xincmm]

@SJLNNU, @aditya-malte I also encountered the same problem. I solved it. the output is a tensor, tensor and PIL can be converted。in the val.py add the following cod:

from torchvision import transforms
import matplotlib.pyplot as plt

loader = transforms.Compose([
    transforms.ToTensor()])  

unloader = transforms.ToPILImage()

def imshow(tensor, title=None):
    image = tensor.cpu().clone()  # we clone the tensor to not do changes on it
    image = image.squeeze(0)  # remove the fake batch dimension
    image = unloader(image)
    plt.imshow(image)
    if title is not None:
        plt.title(title)
    plt.pause(0.001)  # pause a bit so that plots are updated

Now use can use imshow like this: imshow(output) result:

And then，you can also save tensor or PIL to image by use torchvision.utils.save_image reference: https://pytorch.org/docs/stable/torchvision/utils.html

[SJLNNU]

@SJLNNU, @aditya-malte I also encountered the same problem. I solved it. the output is a tensor, tensor and PIL can be converted。in the val.py add the following cod:

from torchvision import transforms
import matplotlib.pyplot as plt

loader = transforms.Compose([
    transforms.ToTensor()])  

unloader = transforms.ToPILImage()

def imshow(tensor, title=None):
    image = tensor.cpu().clone()  # we clone the tensor to not do changes on it
    image = image.squeeze(0)  # remove the fake batch dimension
    image = unloader(image)
    plt.imshow(image)
    if title is not None:
        plt.title(title)
    plt.pause(0.001)  # pause a bit so that plots are updated

Now use can use imshow like this: imshow(output) result:

And then，you can also save tensor or PIL to image by use torchvision.utils.save_image referencer: https://pytorch.org/docs/stable/torchvision/utils.html

Thanks men ,I solved too.

[hiramustafa77]

Hello, The output of the model can be converted to a heatmap by using matplotlib like this:

from matplotlib import cm plt.imshow(output , cmap = cm.jet )

This maps the raw [0,1] to H->[240, 0]. (H being hue from HSV).

PS. For your second doubt: The count values might vary a tiny bit due to the gaussian blur(this is because only an infinitely large kernel can approximate the gaussian blur accurately)

Thanks

Thank you so much for this help.