Huge RAM usage during Evaluation

[NikoRohr]

Hey,

Im using TrackEval for evaluation of the same tracker with different parameter. So I have arround 120 tracker Folder each with to result files of two sequences. Im using run_mot_challenge for evaluation and have the following file structure:

.                                                                                                                                                                     
├── gt                                                                                                                                                                
│   └── penguin                                                                                                                                                       
│       ├── MOT17-train                                                                                                                                               
│       │   ├── EQ0A4452                                                                                                                                              
│       │   │   ├── gt                                                                                                                                                
│       │   │   │   └── gt.txt                                                                                                                                        
│       │   │   └── seqinfo.ini                                                                                                                                       
│       │   └── EQ0A4453                                                                                                                                              
│       │       ├── gt                                                                                                                                                
│       │       │   └── gt.txt                                                                                                                                        
│       │       └── seqinfo.ini                                                                                                                                       
│       └── seqmaps                                                                                                                                                   
│           └── MOT17-train.txt                                                                                                                                       
└── trackers                                                                                                                                                          
    └── penguin                                                                                                                                                       
        └── MOT17-train                                                                                                                                               
            ├── sort_0_10_0.5_man
            │   └── data
            │       ├── EQ0A4452.txt
            │       └── EQ0A4453.txt
            └── sort_5_8_0.75_man
                └── data
                    ├── EQ0A4452.txt
                    └── EQ0A4453.txt
...

Everything is working fine but the Memory usage goes clearly above 20GB. Even with a swap of 30GB it is not enough. But I have to mention that not every evaluation consumes so much memory. 30 runs or something like this needed less than 20 GB

The biggest directory is 65MB and the ground truth is in for both sequences 25MB.

I am really wondering why so much memory is used and hope someone can help me to fix the problem.

[ohjho]

saw similar issues when evaluating on the SportsMOT dataset

I found that if you set --USE_PARALLEL True and --NUM_PARALLEL_CORE to a reasonable number (i used only 2 because i'm running on an instance with low specs) the memory problem goes away somehow.

for reference, my instance only have 4GB of RAM and I was able to finish the eval script comparing 6 different trackers