nwojke / deep_sort

Simple Online Realtime Tracking with a Deep Association Metric
GNU General Public License v3.0
5.16k stars 1.45k forks source link

Feature extractor cannot be used for Person ReIdentification #285

Open lakshaydulani opened 2 years ago

lakshaydulani commented 2 years ago

Firstly, I am sorry for posting this as an issue.

I am trying to build a Multi Camera Person Tracking system. I want to save and utilize the features extracted by one camera in footage from other cameras.

The Feature extractor which is trained on Mars dataset, doesn't seem to help in differentiating between two different people.

I wrote the below snippet to check the Cosine Distance between images from same person and different person.

`extr = Extractor("./deep_sort_pytorch/deep_sort/deep/checkpoint/ckpt.t7") list = glob.glob("mars/*.jpg") features_list = []

for i in list: im = cv2.imread(i) im_crops = [im] features = extr(im_crops) features_list.append(features)

for f in features_list: print(_cosine_distance(f, features_list[0]),"<<")

def _cosine_distance(a, b, data_is_normalized=False): cos = nn.CosineSimilarity(dim=1, eps=1e-6) return (1. - cos(torch.from_numpy(a), torch.from_numpy(b)))`

As expected, the cosine distance between images of same person is very low. But unexpectedly the cosine distance between crops of two different people is also similarly low.

I thought the Feature extractor will help me in differentiating.

Shall I increase the latent space dimensions from 512 to a bigger size?

Or may be I am mistaking the role of Feature extractor. Please guide.

danielcrane commented 2 years ago

Could it not just be that the dataset the model was trained on isn't very compatible with your task (due to difference in angle, camera quality, etc.)? Is it possible for you to create a dataset of your own and train (or fine-tune) the model?