Python tool generate_aitod_imgs.py doesn't generate a combined AI-TOD and xView dataset with JSON annotations for use in MMDET-AITOD training

[hytel]

I was able to use the AI-TOD V2 Dataset with MMDET-AITOD training by referencing the AI-TOD V2 JSON annotations. All worked well. Model does converge, and inferences look good. But you describe using the 'generate_aitod_imgs.py' tool to create a full AI-TOD AND xView dataset. I assume this is to get an even bigger dataset? But, when I run the tool, it creates many directories, but the only new directory that looks like the full set is the 'xview_aitod_sets' directory in the xview folder, not the aitod folder. This isn't what the tool description is. Anyway, the problem is that the 'xview_aitod_sets' folder does have what looks like a full set of data (train, val, trainval, and test), but it's NOT in JSON format like AI-TOD. It's in images/labels format (text files which correspond to image files) which is more like YOLO. There is no annotations JSON file with AI-TOD and xView labels together.

This is NOT what your MMDET-AITOD is seems to be looking for in the 'mmdet-aitod/mmdet-nwdrka/configs_nwdrka/nwd_rka' V2 config files. They are looking for JSON annotation files.

Questions:

1. Is AI-TOD just the vehicles from xView, or is it a different dataset that can be COMBINED with xView vehicles it using your tool?
2. If it is a different dataset, then why is the full set that is written out when running your 'generate_aitod_imgs.py' tool not in the AI-TOD format so it can also be used with MMDET-AITOD?

[Chasel-Tsui]

Hi, thanks for your interest in our work. First of all, we would like to clarify that we provide the 'generate_aitod_imgs.py' tool to create a full AI-TOD dataset, since the AI-TOD contains some images from xview. But we were informed by the official that we cannot directly release the images of xview otherwise there would be a license conflict, so we remove the images of xview and provide the part of AI-TOD w/o xview for the users to generate the full AI-TOD. However, there are some websites that release the AI-TOD dataset like this_one, you can actually download it there... Besides, the txt files are middle files when generating the complete dataset, the final correct annotation files are in the .json format

[hytel]

Thanks for responding right away! I did suspect the xView data was kept separately due to licensing, but I am very interested in the combined version. AITOD + xView. The problem is the JSON file ISN'T being written anywhere I can see by the 'generate_aitod_imgs.py' tool. In fact, I don't even see where in the script your WRITE ANY JSON file other than in functions that aren't being called (coco_merge, XVIEW2COCO). All I have is the images/labels directories. In order to do the training with the MMDET-AITOD code base though, it looks like I need the JSON annotations file. How did you generate it? The script seems incomplete. It ends without even doing any 'cleanup' even though the comment is there?

[hytel]

Also, I did try the link you provided above. Yes, after going through the registering process, and downloading 22gb of data, it looks like this is just the full v1 dataset (probably including xView) - which you point out very explicitly in the AI-TOD V2 site that it is MUCH worse than the v2 dataset which I am trying to construct using your tool. Again, it does look like in produces a full set of AITOD V2 images/labels, but no JSON annotations file. Is there a v2 zip downloadable version with JSON annotations?

[Chasel-Tsui]

Hi, sorry for the ambiguity, the 'generate_aitod_imgs.py' is only used to generate the images of AI-TOD. The AI-TOD and AI-TOD-v2 share images but have different annotations. We directly release the whole annotations of AI-TOD and AI-TOD-v2 in the .json format. You can download the AI-TOD's annotation here or AI-TOD-v2's annotation here

[hytel]

Right, but, when you say "AI-TOD-v2's annotations", do you mean AI-TOD + xView? Because When I create the "full" set, I need annotations for BOTH in JSON, right? My impression is that the script does TWO things, though it isn't clear. It creates a new custom dataset, that is both 1. The combined set of AI-TOD V2 + xView, and 2. Each image is sub-cropped into tine subimage_size x subimage_size. If so, the new dataset that is formed needs a NEW JSON annotations file, right?

[Chasel-Tsui]

AI-TOD-v2 refers to this paper where we re-annotate the AI-TOD and release a version of much higher annotation quality. The AI-TOD-v2 is not a combination of AI-TOD+ xview, the AI-TOD-v2 shares the same images with AI-TOD. If you want to use AI-TOD-v2, the simplest way is to download the AI-TOD's images and AI-TOD-v2's annotations from the above links.

[hytel]

Yup, I did. But as I say, I would like the AI-TOD V2 + xView dataset merged (because it's a larger dataset!) as you point out in your github page and thus the Python tool described above. The problem, is the tool above doesn't produce the COMBINED JSON annotations file does it? I do have the geojson from xView. I do have the JSON annotations for AI-TOD V2 in the link above. I can run your Python script and produce what appears to be a COMBINED AI-TOD V2 + xView dataset chipped to subimage_size x subimage_size. The problem is there is no corresponding JSON annotations file describing this new full set. But, this JSON annotations file is needed for the next step: Training on this new COMBINED dataset, right?

[Chasel-Tsui]

Yes, the provided tool cannot generate a combined AI-TOD V2 + xView. It is interesting to merge the AI-TOD V2 with xview to get a larger dataset but you need to write new codes for that.

[hytel]

Ok, now I'm really confused. At the top of your main Github page, I read the following:

"[Dataset] Please download the xView trainig set and AI-TOD_wo_xview to construct the complete AI-TOD dataset!"

All I'm trying to do is exactly that! I download both AI-TOD (v2 annotations) and xView training set and I'm trying to use the exact script you mention above to make in your words: "the complete AI-TOD dataset!". Right ?????

[Chasel-Tsui]

I mean there is no need to use the generation tool to generate images or annotations if you just want to use AI-TOD or v2, part of the xview is already included in the AI-TOD series, you can directly download images and annotations from the above mentioned links. But as you mentioned, if you want to generate AI-TOD+xview, this repo will not be helpful for you since AI-TOD+xview is totally a new dataset different from our published datasets.

[hytel]

Confused again. You say to get there's AITOD and xView. And you need the tool to create the full set. But, somehow "part of the xview is already included" just in AITOD? Which part? Where is this explained? My impression is they are separate, and you have to combine them using the tool provided?