search

liangzheng06 / MARS-evaluation

This repository provides the evaluation codes for the MARS dataset
http://www.liangzheng.com.cn/Project/project_mars.html
177 stars 79 forks source link

gallery set contains the videos from the same camera of the query video? #3

Closed YurongYou closed 6 years ago

YurongYou commented 7 years ago

Hi!

From the evaluation protocol part of your paper, it is said that

query and gallery are captured by different cameras.

But from https://github.com/liangzheng06/MARS-evaluation/blob/3a91bbc762fad7629fecde8c53e170742d3700f9/utils/evaluation_mars.m#L12-L18

it does not mark the feature of persons of a different label but with the same camera as ``junk'', which will make the computed CMC lower than using the setting of cross-camera search.

liangzheng06 commented 7 years ago

Hi,

What you suggest is actually correct for evaluation. It accords precisely to what I described in my paper.

But in the current evaluation, I would rather keep the persons in the same query camera in the database. It helps simplify the code and enlarges the actual database.

I think both practices are acceptable.

YurongYou commented 7 years ago

Yes, I agree. Thanks for your prompt reply!

liangzheng06 commented 7 years ago

Sure. I would suggest you use the provided evaluation package so that your results are comparable with other people.