Running the code on new images

[ZeeRizvee]

Hi there! Thank you for sharing the code. Can you please guide me over how can I use this code on my own dataset? I want to give in a few images as input, and want the pre-trained model of fold3 to evaluate those images and output a crowd count. Looking forward to your help. TIA!

[RQuispeC]

Hello,

For pre-trained models refer to #1.

In order to add new datasets you have to add a new class for it on manage_data/dataset_loader.py For instance, considering:

DIR_TRAIN_IMG: path for train image DIR_TRAIN_DEN: path for train density maps DIR_TEST_IMG: path for test image DIR_TEST_DEN: path for test density maps

You can add

class Mytest(object):
    train_test_size = 1
    metadata = dict()
    train_test_set = []
    def __init__(self, force_create_den_maps = False, force_augmentation = False, **kwargs):
        self.metadata = kwargs
        kwargs['name'] = 'test'
        train_test = train_test_unit(DIR_TRAIN_IMG, DIR_TRAIN_DEN , DIR_TEST_IMG, DIR_TEST_DEN, kwargs.copy())
        self.train_test_set.append(train_test)

__factory = {
    'ucf-cc-50': UCF_CC_50,
    'shanghai-tech': ShanghaiTech,
    'mytest': Mytest
}

to manage_data/dataset_loader.py. Notice that when running test (--evaluate-only), only DIR_TEST_IMG and DIR_TEST_DEN are used, so you can leave DIR_TRAIN_IMG="", DIR_TRAIN_DEN=""

After that you use your new dataset using the parameter -d

python3 train.py -d mytest --save-dir log/ --evaluate-only --resume models/best_model_fold3.h5 --save-plots

That should make it

[ZeeRizvee]

Thank you for the timely response. Do I need to provide the density maps for my test images too? I'm confused since plotting the generated density maps for the UCF dataset shows they are of the ground truth images. I do not need the MAE and MSE scores at the end. Only the estimated crowd count is required. So can I work only with the test images in this case?

[RQuispeC]

If you just want to generate the density maps, you don't need the ground truths. In that case you'll have to modify architecture/evaluate_model.py accordingly

[ZeeRizvee]

Making changes in the evaluate_model.py script only visually removes the ground truth images on the plot. It seems that in some other script the program is still comparing it with the ground truths. I'm trying to understand the flow of the program, but one thing is confusing me. In the actual training and testing with the UCF dataset, the ground truth density maps were provided. When testing on new images, will the program still be requiring ground truth density maps and json files for each new image for comparing, or will the predicted density maps made at run-time suffice to give a crowd count? How do I populate the 'gt_density' blob in data_loader.py since there will not be any ground truth densities available in my case. Looking forward to your kind assistance.

[RQuispeC]

Hi ZeeRizvee,

I reviewed your comment and realize that my previous answer was not complete! sorry for that.

To test on your own dataset, you also need to update data_loader.py . data_loader.py loads the data for train and test, in your case you may have to edit function __iter__ in this section:

if self.test_loader: #loader is for testing, then we divide the image in chunks of size (img_height, img_width)
                    _, h, w = img.shape
                    orig_shape = (h, w)
                    # compute padding
                    if self.test_overlap:
                        padding_h = self.img_height - max(h % self.img_height, (h - self.img_height//2) % self.img_height)
                        padding_w = self.img_width - max(w % self.img_width, (w - self.img_width//2) % self.img_width)
                    else:
                        padding_h = self.img_height - (h % self.img_height)
                        padding_w = self.img_width - (w % self.img_width)

                    # add padding
                    img = np.concatenate((img, np.zeros((img.shape[0], padding_h, img.shape[2]))), axis =1)
                    den = np.concatenate((den, np.zeros((img.shape[0], padding_h, img.shape[2]))), axis =1)
                    img = np.concatenate((img, np.zeros((img.shape[0], img.shape[1], padding_w))), axis =2)
                    den = np.concatenate((den, np.zeros((img.shape[0], img.shape[1], padding_w))), axis =2)
                    assert img.shape[1] % 2 == 0 and img.shape[2] % 2 == 0, "Inputs images must have even dimensions, found {}".format(img.shape)

                    # create batch for test
                    _, h, w = img.shape
                    new_shape = (h, w)
                    disp_height = self.img_height // 2 if self.test_overlap else self.img_height
                    disp_width = self.img_width // 2 if self.test_overlap else self.img_width
                    for i in range(0, h - self.img_height + 1, disp_height):
                        for j in range(0, w - self.img_width + 1, disp_width):
                            chunk_img = img[0, i:i + self.img_height, j:j + self.img_width]
                            chunk_den = den[0, i:i + self.img_height, j:j + self.img_width]
                            chunk_img = chunk_img.reshape((1, chunk_img.shape[0], chunk_img.shape[1]))
                            chunk_den = chunk_den.reshape((1, chunk_den.shape[0], chunk_den.shape[1]))
                            imgs.append(chunk_img)
                            dens.append(chunk_den)

basically you need to remove any reference to variables den, chunk_den or dens, and populate variable blob['gt_density'] with empty value. Finally, you also need to update function recontruct_test similarly to __iter__.

[ZeeRizvee]

I was able to grasp the program flow yesterday, and i'm happy to see that the changes suggested by you align with the ones I had made after my understanding. For plotting the results however, architecture/utils.py also has to be edited, and every reference to gt_density_map has to be removed. Furthermore, I had completely removed blob['gt_density'] from data_loader.py and then also removed the corresponding variable gt_data from evaluate_model.py. The code seems to be working perfectly now to give my desired output! Thank you for your time and assistance. Cheers!