search

MCG-NJU / MeMOTR

[ICCV 2023] MeMOTR: Long-Term Memory-Augmented Transformer for Multi-Object Tracking
https://arxiv.org/abs/2307.15700
MIT License
140 stars 8 forks source link

How to fine tune a custom VOC Dataset? #9

Closed danial880 closed 7 months ago

danial880 commented 8 months ago

Thanks for your amazing work. I have a dataset labeled in voc. It has a directory structure as follows:

Dataset |____video1 ------------------|__img1 ------------------|__img1.xml |____video2 ------------------|__img1 ------------------|__img1.xml

I have written a custom script for converting into coco video dataset.

HELLORPG commented 8 months ago

Thanks for your interest in our research.

I'm not very familiar with the VOC format or the COCO format, as we use the standard MOT format in our dataset (you can refer to the paper of MOT17 or the repository of DanceTrack). For each video sequence, the files should be organized as following structure:

{Data ROOT}
|-- Your Video Dataset Name
|   |-- train
|   |   |-- video sequence 1
|   |   |   |-- img1
|   |   |   |   |-- 00000001.jpg
|   |   |   |   |-- ...
|   |   |   |-- gt
|   |   |   |   |-- gt.txt
|   |   |-- ...
|   |-- val
|   |   |-- ...
|   |-- test
|   |   |-- ...
|   |-- train_seqmap.txt
|   |-- val_seqmap.txt
|   |-- test_seqmap.txt
|-- ...

As for a specific gt.txt, it contains all annotations for each frame in this sequence. The annotations are ordered in the frame index, each line is like follows:

<frame>, <id>, <bb_left>, <bb_top>, <bb_width>, <bb_height>, 1, 1, 1

As you already have the videos in the COCO format, I think that you should write an additional script to convert coco into the above mot format.

However, directly using the data in the COCO format for class Dataset may be a better choice as many datasets have a convert_to_coco script. If I have some other spare time, I might consider adding this feature. Cause as you can see, I use different Dataset classes for different training datasets (MOT17/DanceTrack), this is a little inelegant.

danial880 commented 8 months ago

Thanks for your reply.

How to evaluate my custom model. My dataset was not in coco classes so I have trained a custom dab-deformable detr for this training. I have written a script for converting VOC to dancetrack. The model is performing well, I have checked it with demo.ipynb. I am unable to evaluate my custom model. Here is the traceback: no seqmap found: ./datasets/DanceTrack/val_seqmap.txt Traceback (most recent call last): File "/home/danial/Desktop/MeMOTR/TrackEval/scripts/run_mot_challenge.py", line 84, in <module> dataset_list = [trackeval.datasets.MotChallenge2DBox(dataset_config)] File "/home/danial/Desktop/MeMOTR/TrackEval/trackeval/datasets/mot_challenge_2d_box.py", line 82, in __init__ self.seq_list, self.seq_lengths = self._get_seq_info() File "/home/danial/Desktop/MeMOTR/TrackEval/trackeval/datasets/mot_challenge_2d_box.py", line 157, in _get_seq_info raise TrackEvalException('no seqmap found: ' + os.path.basename(seqmap_file)) trackeval.utils.TrackEvalException: no seqmap found: val_seqmap.txt Traceback (most recent call last): File "/home/danial/Desktop/MeMOTR/main.py", line 120, in <module> main(config=merged_config) File "/home/danial/Desktop/MeMOTR/main.py", line 107, in main evaluate(config=config) File "/home/danial/Desktop/MeMOTR/eval_engine.py", line 36, in evaluate metrics = eval_model(model=config["EVAL_MODEL"], eval_dir=eval_dir, File "/home/danial/Desktop/MeMOTR/eval_engine.py", line 118, in eval_model with open(metric_path) as f: FileNotFoundError: [Errno 2] No such file or directory: './outputs/memotr_dancetrack/val/_tracker/pedestrian_summary.txt'

It is looking for val_seqmap.txt which I am unable to find in official dancetrack dataset downloads that is why I am unable to replicate it for my dataset. Secondly, it is looking for pedestrian_summary.txt but I haven't trained it on pedestrians.

HELLORPG commented 8 months ago

Oh, the val_seqmap.txt can be obtained from the official repo of DanceTrack (it is a sequence name list of your data split). You should put this file as we showed here in our docs.

And for the pedestrian_summary.txt, if the evaluation code has been run well, it will automatically generate in the tracking results' dir, containing the evaluation result, you should not worry about that.

danial880 commented 7 months ago

Where to find this file or how I can produce this for my dataset: ./outputs/memotr_dancetrack/val/checkpoint_19_tracker/dancetrack0001.txt Traceback Tracker file not found: ./outputs/memotr_dancetrack/val/checkpoint_19_tracker/dancetrack0001.txt Traceback (most recent call last): File "/home/danial/Desktop/MeMOTR/TrackEval/scripts/run_mot_challenge.py", line 84, in <module> dataset_list = [trackeval.datasets.MotChallenge2DBox(dataset_config)] File "/home/danial/Desktop/MeMOTR/TrackEval/trackeval/datasets/mot_challenge_2d_box.py", line 124, in __init__ raise TrackEvalException( trackeval.utils.TrackEvalException: Tracker file not found: //dancetrack0001.txt Traceback (most recent call last): File "/home/danial/Desktop/MeMOTR/main.py", line 120, in <module> main(config=merged_config) File "/home/danial/Desktop/MeMOTR/main.py", line 107, in main evaluate(config=config) File "/home/danial/Desktop/MeMOTR/eval_engine.py", line 36, in evaluate metrics = eval_model(model=config["EVAL_MODEL"], eval_dir=eval_dir, File "/home/danial/Desktop/MeMOTR/eval_engine.py", line 118, in eval_model with open(metric_path) as f: FileNotFoundError: [Errno 2] No such file or directory: './outputs/memotr_dancetrack/val/checkpoint_19_tracker/pedestrian_summary.txt'

HELLORPG commented 7 months ago

Are you trying to evaluate the model? This error is because the corresponding GT file was not found during the evaluation.

danial880 commented 7 months ago

Yes, I am trying to evaluate the model.

python main.py --mode eval --data-root ./datasets --eval-mode specific --eval-model checkpoint_19.pth --eval-dir ./outputs/memotr_dancetrack/ --eval-threads 0

My dataset directory

datasets/DanceTrack/val
├── dancetrack0001
│   ├── gt
│   │   └── gt.txt
│   ├── img1
│   │   ├── 00000001.jpg
│   │   ├── 00000002.jpg
│   │   ├── 00000003.jpg
│   │   ├── 00000004.jpg
│   │   ├── 00000005.jpg
│   │   ├── 00000006.jpg
│   │   ├── 00000007.jpg
│   │   ├── 00000008.jpg
│   │   ├── 00000009.jpg
│   │   ├── 00000010.jpg
│   │   ├── 00000011.jpg
│   │   ├── 00000012.jpg
│   │   ├── 00000013.jpg
│   │   ├── 00000014.jpg
│   │   ├── 00000015.jpg
│   │   ├── 00000016.jpg
│   │   ├── 00000017.jpg
│   │   ├── 00000018.jpg
│   │   ├── 00000019.jpg
│   │   ├── 00000020.jpg
│   │   ├── 00000021.jpg
│   │   ├── 00000022.jpg
│   │   ├── 00000023.jpg
│   │   ├── 00000024.jpg
│   │   ├── 00000025.jpg
│   │   ├── 00000026.jpg
│   │   ├── 00000027.jpg
│   │   ├── 00000028.jpg
│   │   └── 00000029.jpg
│   └── seqinfo.ini
├── dancetrack0002
│   ├── gt
│   │   └── gt.txt
│   ├── img1
│   │   ├── 00000001.jpg
│   │   ├── 00000002.jpg
│   │   ├── 00000003.jpg
│   │   ├── 00000004.jpg
│   │   ├── 00000005.jpg
│   │   ├── 00000006.jpg
│   │   ├── 00000007.jpg
│   │   ├── 00000008.jpg
│   │   ├── 00000009.jpg
│   │   ├── 00000010.jpg
│   │   ├── 00000011.jpg
│   │   ├── 00000012.jpg
│   │   ├── 00000013.jpg
│   │   ├── 00000014.jpg
│   │   ├── 00000015.jpg
│   │   ├── 00000016.jpg
│   │   ├── 00000017.jpg
│   │   ├── 00000018.jpg
│   │   ├── 00000019.jpg
│   │   ├── 00000020.jpg
│   │   ├── 00000021.jpg
│   │   ├── 00000022.jpg
│   │   ├── 00000023.jpg
│   │   ├── 00000024.jpg
│   │   ├── 00000025.jpg
│   │   ├── 00000026.jpg
│   │   ├── 00000027.jpg
│   │   ├── 00000028.jpg
│   │   ├── 00000029.jpg
│   │   └── 00000030.jpg
│   └── seqinfo.ini
├── dancetrack0003
│   ├── gt
│   │   └── gt.txt
│   ├── img1
│   │   ├── 00000001.jpg
│   │   ├── 00000002.jpg
│   │   ├── 00000003.jpg
│   │   ├── 00000004.jpg
│   │   ├── 00000005.jpg
│   │   ├── 00000006.jpg
│   │   ├── 00000007.jpg
│   │   ├── 00000008.jpg
│   │   ├── 00000009.jpg
│   │   ├── 00000010.jpg
│   │   ├── 00000011.jpg
│   │   ├── 00000012.jpg
│   │   ├── 00000013.jpg
│   │   ├── 00000014.jpg
│   │   ├── 00000015.jpg
│   │   ├── 00000016.jpg
│   │   ├── 00000017.jpg
│   │   ├── 00000018.jpg
│   │   ├── 00000019.jpg
│   │   ├── 00000020.jpg
│   │   ├── 00000021.jpg
│   │   └── 00000022.jpg
│   └── seqinfo.ini
├── dancetrack0004
│   ├── gt
│   │   └── gt.txt
│   ├── img1
│   │   ├── 00000001.jpg
│   │   ├── 00000002.jpg
│   │   ├── 00000003.jpg
│   │   ├── 00000004.jpg
│   │   ├── 00000005.jpg
│   │   ├── 00000006.jpg
│   │   ├── 00000007.jpg
│   │   ├── 00000008.jpg
│   │   ├── 00000009.jpg
│   │   ├── 00000010.jpg
│   │   ├── 00000011.jpg
│   │   ├── 00000012.jpg
│   │   ├── 00000013.jpg
│   │   ├── 00000014.jpg
│   │   ├── 00000015.jpg
│   │   ├── 00000016.jpg
│   │   ├── 00000017.jpg
│   │   ├── 00000018.jpg
│   │   ├── 00000019.jpg
│   │   ├── 00000020.jpg
│   │   └── 00000021.jpg
│   └── seqinfo.ini
├── dancetrack0005
│   ├── gt
│   │   └── gt.txt
│   ├── img1
│   │   ├── 00000001.jpg
│   │   ├── 00000002.jpg
│   │   ├── 00000003.jpg
│   │   ├── 00000004.jpg
│   │   ├── 00000005.jpg
│   │   ├── 00000006.jpg
│   │   ├── 00000007.jpg
│   │   ├── 00000008.jpg
│   │   ├── 00000009.jpg
│   │   ├── 00000010.jpg
│   │   ├── 00000011.jpg
│   │   ├── 00000012.jpg
│   │   ├── 00000013.jpg
│   │   ├── 00000014.jpg
│   │   ├── 00000015.jpg
│   │   ├── 00000016.jpg
│   │   ├── 00000017.jpg
│   │   ├── 00000018.jpg
│   │   └── 00000019.jpg
│   └── seqinfo.ini
├── dancetrack0006
│   ├── gt
│   │   └── gt.txt
│   ├── img1
│   │   ├── 00000001.jpg
│   │   ├── 00000002.jpg
│   │   ├── 00000003.jpg
│   │   ├── 00000004.jpg
│   │   ├── 00000005.jpg
│   │   ├── 00000006.jpg
│   │   ├── 00000007.jpg
│   │   ├── 00000008.jpg
│   │   ├── 00000009.jpg
│   │   ├── 00000010.jpg
│   │   ├── 00000011.jpg
│   │   ├── 00000012.jpg
│   │   ├── 00000013.jpg
│   │   ├── 00000014.jpg
│   │   ├── 00000015.jpg
│   │   └── 00000016.jpg
│   └── seqinfo.ini
└── dancetrack0007
    ├── gt
    │   └── gt.txt
    ├── img1
    │   ├── 00000001.jpg
    │   ├── 00000002.jpg
    │   ├── 00000003.jpg
    │   ├── 00000004.jpg
    │   ├── 00000005.jpg
    │   ├── 00000006.jpg
    │   ├── 00000007.jpg
    │   ├── 00000008.jpg
    │   ├── 00000009.jpg
    │   ├── 00000010.jpg
    │   ├── 00000011.jpg
    │   ├── 00000012.jpg
    │   ├── 00000013.jpg
    │   ├── 00000014.jpg
    │   ├── 00000015.jpg
    │   ├── 00000016.jpg
    │   ├── 00000017.jpg
    │   ├── 00000018.jpg
    │   ├── 00000019.jpg
    │   └── 00000020.jpg
    └── seqinfo.ini

I have val_seqmap.txt inside the DanceTrack Folder (same directory as the val folder). Screenshot from 2024-01-31 16-38-34

HELLORPG commented 7 months ago

Sorry, I made a mistake just now. This error indicates that you do not have the tracking results file dancetrack0001.txt.

May I see the file list in ./outputs/memotr_dancetrack/val/checkpoint_19_tracker/ folder?

danial880 commented 7 months ago

There is no such folder like checkpoint_19_tracker in my ./outputs/memotr_dancetrack/val. Here is my val folder:

val
├── config.yaml
├── tb
│   ├── events.out.tfevents.1703921038.lvision.8136.0
│   ├── events.out.tfevents.1703922477.lvision.8782.0
│   ├── events.out.tfevents.1703949337.lvision.10723.0
│   ├── events.out.tfevents.1706627493.lvision.9255.0
│   └── events.out.tfevents.1706628478.lvision.10394.0
├── tb_epochs_log
│   ├── events.out.tfevents.1703921039.lvision.8140.1
│   ├── events.out.tfevents.1703922478.lvision.8786.1
│   ├── events.out.tfevents.1703949338.lvision.10727.1
│   ├── events.out.tfevents.1706627495.lvision.9259.1
│   └── events.out.tfevents.1706628479.lvision.10398.1
└── tb_iters_log
    ├── events.out.tfevents.1703921039.lvision.8140.0
    ├── events.out.tfevents.1703922478.lvision.8786.0
    ├── events.out.tfevents.1703949338.lvision.10727.0
    ├── events.out.tfevents.1706627495.lvision.9259.0
    └── events.out.tfevents.1706628479.lvision.10398.0

My training command:

python main.py --config-path ./configs/train_dancetrack.yaml --outputs-dir ./outputs/memotr_dancetrack/ --batch-size 1 --data-root ./datasets --pretrained-model checkpoint0599.pth

How can I generate this, is it generated during training or there is some other script?

HELLORPG commented 7 months ago

This is strange because if you run an evaluation, there should be a tracker folder where the output tracking results are stored. The related code is here, and the outputs dir is decided here.

I think you may first need to find where the output results are. Maybe you can only run the submit script to find if the tracking results have been saved.

danial880 commented 7 months ago

Hi, just figured out my mistake, I have kept the checkpoint.pth outside the eval directory. Now it is evaluating. Thanks for your help.