python api

[mosheliv]

do you have python api like mmdetection that easily allows you to configure new dataset and modify configuration using code?

in mmdetection you can do something like:

@DATASETS.register_module()
class XXXDataset(CustomDataset):

CLASSES = ("ignored regions", ...., "others")                                             

def load_annotations(self, ann_file):     

. .

and modify the config like this: cfg = Config.fromfile('./configs/fp16/faster_rcnn_r50_fpn_fp16_1x_coco.py')
cfg.workflow=[('train',1),('val',1)]

you code structure seems very similar but i couldn't find this functionality.

[qingqing01]

You can refer COCODataSet, register your custom dataset:

from ppdet.core.workspace import register, serializable
from .dataset import DetDataset

@register
@serializable
class XXXDataSet(DetDataset):

Then, config the XXXDataset in the YML config, like:

https://github.com/PaddlePaddle/PaddleDetection/blob/d7383ad99c69e03f984ead52cc645d17f4729837/configs/datasets/coco_detection.yml#L4-L19

[mosheliv]

Thank you for your prompt reply. This answers only partially my question. Perhaps I wasn't clear enough. What I want to do is:

1. In a script, say "moshe.py",define the dataset (you answered this, thanks)
2. Modify the config
3. Call the trainings.

On Tue, Jul 20, 2021, 01:21 qingqing01 @.***> wrote:

You can refer COCODataSet, register your custom dataset:

from ppdet.core.workspace import register, serializable from .dataset import DetDataset

@register @serializable class XXXDataSet(DetDataset):

Then, config the XXXDataset in the YML config, like:

https://github.com/PaddlePaddle/PaddleDetection/blob/d7383ad99c69e03f984ead52cc645d17f4729837/configs/datasets/coco_detection.yml#L4-L19

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/PaddlePaddle/PaddleDetection/issues/3697#issuecomment-882542563, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC7IWCZCZ6UTFWY4ONVLKDLTYQREDANCNFSM5AMLUZUQ .

[qingqing01]

2. Modify the config

You can choose one model config, for example:

configs/faster_rcnn/faster_rcnn_r50_1x_coco.yml:

https://github.com/PaddlePaddle/PaddleDetection/blob/d7383ad99c69e03f984ead52cc645d17f4729837/configs/faster_rcnn/faster_rcnn_r50_1x_coco.yml#L1-L8

the dataset config in this faster_rcnn_r50_1x_coco.yml is here:

https://github.com/PaddlePaddle/PaddleDetection/blob/d7383ad99c69e03f984ead52cc645d17f4729837/configs/datasets/coco_detection.yml#L4-L19

Then modify the dataset config.

You also can copy these config in configs/faster_rcnn/faster_rcnn_r50_1x_coco.yml to modify.

3. Call the trainings.

Then train by tools/train.py

[mosheliv]

Thank you, I know that. in mmdetection I have a training file that looks like this:

############## CODE START from mmdet.datasets.builder import DATASETS

from mmdet.datasets.custom import CustomDataset

from mmdet.apis import set_random_seed

import cv2

@DATASETS.register_module()

class XXXDataset(CustomDataset): . . .

from mmcv import Config

cfg =

Config.fromfile('./configs/fcos/fcos_center-normbbox-centeronreg-giou_r50_caffe_fpn_gn-head_dcn_1x_coco.py')

cfg = Config.fromfile('./configs/fp16/faster_rcnn_r50_fpn_fp16_1x_coco.py')

cfg.workflow=[('train',1),('val',1)] cfg.dataset_type = 'XXXDataset' . . .

from mmdet.datasets import build_dataset

from mmdet.models import build_detector

from mmdet.apis import train_detector

Build dataset

datasets = [build_dataset(cfg.data.train), build_dataset(cfg.data.val)]

Build the detector

model = build_detector(

cfg.model, train_cfg=cfg.get('train_cfg'),

test_cfg=cfg.get('test_cfg'))

Add an attribute for visualization convenience

model.CLASSES = datasets[0].CLASSES

Create work_dir

mmcv.mkdir_or_exist(osp.abspath(cfg.work_dir))

train_detector(model, datasets, cfg, distributed=False, validate=True)

#################### CODE END

Can something similar be done in PaddlePaddle? For complicated reasons fiddling with the configuration files directly won't be good idea.

On Fri, Jul 23, 2021, 21:58 qingqing01 @.***> wrote:

1. Modify the config

You can choose one model config, for example:

configs/faster_rcnn/faster_rcnn_r50_1x_coco.yml:

https://github.com/PaddlePaddle/PaddleDetection/blob/d7383ad99c69e03f984ead52cc645d17f4729837/configs/faster_rcnn/faster_rcnn_r50_1x_coco.yml#L1-L8

the dataset config in this faster_rcnn_r50_1x_coco.yml is here:

https://github.com/PaddlePaddle/PaddleDetection/blob/d7383ad99c69e03f984ead52cc645d17f4729837/configs/datasets/coco_detection.yml#L4-L19

Then modify the dataset config.

You also can copy these config in configs/faster_rcnn/faster_rcnn_r50_1x_coco.yml to modify.

1. Call the trainings.

Then train by tools/train.py

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/PaddlePaddle/PaddleDetection/issues/3697#issuecomment-885530523, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC7IWCZWGWSKGI52LKLMW53TZE4NTANCNFSM5AMLUZUQ .