search

cheind / py-motmetrics

:bar_chart: Benchmark multiple object trackers (MOT) in Python
MIT License
1.37k stars 259 forks source link

How to get MOT metrics on custom test set? #131

Open MKiremitci opened 3 years ago

MKiremitci commented 3 years ago

Hi everyone and @cheind,

Thank you for this great jobs. I am newbie and I trained an object detector to detect people from top-view. Now, I want to deal with Multiple Object Tracking, but I don't know how can I get the MOT results. I think lots of people are using the MOT Challange's datasets to evaluate their Multiple Object tracking methods. However, when I look to MOT Challanges, there is no top-view people datasets to evaluate my method. Also, I have already gather top-view human videos to use with tracking.

Therefore, I want to know how should I prepare the test set? How should I annotate? Should test set contain images or videos?

If you answer these questions with basic language to me, I will be too happy. Thanks for your help.

khalidw commented 3 years ago

@MKiremitci Hi! I am also working on a similar problem. I have a custom made dataset of boats for which I am trying to calculate MOT metrics using two different tracking algorithm.

I have annotated the dataset myself, as I understand for custom dataset you need to manually create a ground truth file as per MOT16 or equivalent format.

I am generating a text file of all the tracks in MOT16 format for each tracker.

Once I have both ground truth and tracker files, I use them to generate the metric. I hope this is helpful to you

cheind commented 3 years ago

Hey, @MKiremitci you might want to look at the documentation of

https://github.com/cheind/py-motmetrics/blob/develop/motmetrics/apps/eval_motchallenge.py

which gives you an idea of how to structure files so that you can use the script for evaluation.

MKiremitci commented 3 years ago

@khalidw and @cheind Thanks for your answers. :)

ImSuMyatNoe commented 2 years ago

@MKiremitci Hi! I am also working on a similar problem. I have a custom made dataset of boats for which I am trying to calculate MOT metrics using two different tracking algorithm.

I have annotated the dataset myself, as I understand for custom dataset you need to manually create a ground truth file as per MOT16 or equivalent format.

I am generating a text file of all the tracks in MOT16 format for each tracker.

Once I have both ground truth and tracker files, I use them to generate the metric. I hope this is helpful to you

@khalidw Hello! I would like to know how to measure with MOT metric on custom dataset! If you have finished the project, could you please let me know how to measure it?

khalidw commented 2 years ago

@ImSuMyatNoe of course. I can share sample code if you can connect on my email address.

ImSuMyatNoe commented 2 years ago

Hello Khalid!

Greetings to you.

I am Su Myat Noe who commented on your github repo related with "[cheind/py-motmetrics] How to get MOT metrics on a custom test set? (#131)". Sorry for taking a long time to reply to your email.

If you have time, could you please share about the " motometrics library" to my email . @.***

Thank you very much for your time and consideration.

Have a nice day!

Best regards, Su Myat Noe

On Sun, Mar 20, 2022 at 2:16 AM Khalid Waleed @.***> wrote:

@ImSuMyatNoe https://github.com/ImSuMyatNoe of course. I have used motmetrics library for that. I can share sample code if you can connect on my email address.

— Reply to this email directly, view it on GitHub https://github.com/cheind/py-motmetrics/issues/131#issuecomment-1073047377, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEWYCEAPFHSVITF2LAPC2ULVAYDXLANCNFSM4UZLI5CA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

khalidw commented 2 years ago

@ImSuMyatNoe Please see your gmail inbox. I have forwarded you the ipynb notebook. If you have any queries do let me know over email

myuvarajmp commented 2 years ago

@khalidw could you please share the notebook to me as well. mail id:m.yuvarajmp@gmail.com

cheind commented 2 years ago

@khalidw if that notebook is of potential interest to many people, would you mind making it publicly available? I'd be happy to put a link to it online

khalidw commented 2 years ago

@cheind Yes I would. Will share in a short while

cheind commented 2 years ago

@khalidw, nice, maybe add PR of readme.md as well? (make sure to base it on develop)

myuvarajmp commented 2 years ago

@khalidw Please share the notebook asap. sorry for the inconvenience.

cheind commented 2 years ago

@myuvarajmp, I've seen @khalidw has already forked the project and is probably already working on a solution. Please be patient.

khalidw commented 2 years ago

@cheind Hi. I have committed a section titled "For custom dataset" on the readme.md. This section has the code and guide on how to use pymotmetrics for custom dataset. Please review and let me know if any changes are required

cheind commented 2 years ago

@myuvarajmp , @everyone would you mind following @khalidw docs on how to use it on custom datasets and check if you run into any problems? Please see #167 for the PR and report there if possible. thanks