Closed lumurillo closed 12 months ago
Use COCO-2017 as an example. Unfortunately, FiftyOne does not support person-keypoints annotations yet.
fiftyone zoo datasets load \
coco-2017 \
-s train \
-n coco-2017-instances-train \
-k include_id=true label_types=detections,segmentations
fiftyone zoo datasets load \
coco-2017 \
-s validation \
-n coco-2017-instances-validation \
-k include_id=true label_types=detections,segmentations
fiftyone datasets create \
--name coco-2017-instances-validation \
--dataset-dir /raid/datasets/coco/ \
--type fiftyone.types.COCODetectionDataset \
--kwargs \
data_path="val2017" \
labels_path=/raid/datasets/coco/annotations/instances_val2017.json \
persistent=true \
include_id=true
datamodule:
train_dataset:
dataset_name: coco-2017-instances-train
gt_field: segmentations
val_dataset:
dataset_name: coco-2017-instances-validation
gt_field: segmentations
FiftyOne commands to load (index) datasets.
Use COCO-2017 as an example. Unfortunately, FiftyOne does not support person-keypoints annotations yet.
Download and load zoo datasets
fiftyone zoo datasets load \ coco-2017 \ -s train \ -n coco-2017-instances-train \ -k include_id=true label_types=detections,segmentations fiftyone zoo datasets load \ coco-2017 \ -s validation \ -n coco-2017-instances-validation \ -k include_id=true label_types=detections,segmentations
Load local datasets
fiftyone datasets create \ --name coco-2017-instances-validation \ --dataset-dir /raid/datasets/coco/ \ --type fiftyone.types.COCODetectionDataset \ --kwargs \ data_path="val2017" \ labels_path=/raid/datasets/coco/annotations/instances_val2017.json \ persistent=true \ include_id=true
Use the FiftyOne datamodule
datamodule: train_dataset: dataset_name: coco-2017-instances-train gt_field: segmentations val_dataset: dataset_name: coco-2017-instances-validation gt_field: segmentations
Is this for the for the example documentation?
What does this PR do?
Add a new datamodule that loads FiftyOne datasets. This implementation let MART delegate the data handling to FiftyOne, where the user can do data curation tasks.
This data handling would be agnostic to MART, because at the end here is adapted a
torchvision
dataset.I set this implementation to be optional. The user can install the fiftyone dependencies by running:
Important:
Type of change
Please check all relevant options.
Testing
Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.
pytest
CUDA_VISIBLE_DEVICES=0 python -m mart experiment=CIFAR10_CNN_Adv trainer=gpu trainer.precision=16
reports 70% (21 sec/epoch).CUDA_VISIBLE_DEVICES=0,1 python -m mart experiment=CIFAR10_CNN_Adv trainer=ddp trainer.precision=16 trainer.devices=2 model.optimizer.lr=0.2 trainer.max_steps=2925 datamodule.ims_per_batch=256 datamodule.world_size=2
reports 70% (14 sec/epoch).Before submitting
pre-commit run -a
command without errorsDid you have fun?
Make sure you had fun coding 🙃