search

AarohiSingla / Key-Point-Detection-on-Custom-Dataset

Keypoint Detection Using Detectron 2 on Custom Dataset
23 stars 6 forks source link

Training on my custom Dataset #1

Open eaedk opened 3 years ago

eaedk commented 3 years ago

Hi, thank you for your sharing. I have an issue when I train on my own dataset. The model is not saved and the keypoint loss stay at 0.

Could you help me please ?

here is the link to the file metric.json

eaedk commented 3 years ago

And I get this output...

[09/13 10:32:14 d2.data.datasets.coco]: Loaded 618 images in COCO format from /content/drive/MyDrive/train_set/train.json [09/13 10:32:14 d2.data.build]: Removed 0 images with no usable annotations. 618 images left. [09/13 10:32:14 d2.data.build]: Removed 0 images with fewer than 1 keypoints. [09/13 10:32:14 d2.data.dataset_mapper]: [DatasetMapper] Augmentations used in training: [ResizeShortestEdge(short_edge_length=(640, 672, 704, 736, 768, 800), max_size=1333, sample_style='choice'), RandomFlip()] [09/13 10:32:14 d2.data.build]: Using training sampler TrainingSampler [09/13 10:32:14 d2.data.common]: Serializing 618 elements to byte tensors and concatenating them all ... [09/13 10:32:14 d2.data.common]: Serialized dataset takes 0.20 MiB WARNING [09/13 10:32:14 d2.solver.build]: SOLVER.STEPS contains values larger than SOLVER.MAX_ITER. These values will be ignored. Skip loading parameter 'roi_heads.keypoint_head.score_lowres.weight' to the model due to incompatible shapes: (512, 17, 4, 4) in the checkpoint but (512, 4, 4, 4) in the model! You might want to double check if this is expected. Skip loading parameter 'roi_heads.keypoint_head.score_lowres.bias' to the model due to incompatible shapes: (17,) in the checkpoint but (4,) in the model! You might want to double check if this is expected. Some model parameters or buffers are not found in the checkpoint: roi_heads.keypoint_head.score_lowres.{bias, weight} [09/13 10:32:14 d2.engine.train_loop]: Starting training from iteration 0 /usr/local/lib/python3.7/dist-packages/numpy/core/fromnumeric.py:3373: RuntimeWarning: Mean of empty slice. out=out, **kwargs) /usr/local/lib/python3.7/dist-packages/numpy/core/_methods.py:170: RuntimeWarning: invalid value encountered in double_scalars ret = ret.dtype.type(ret / rcount) [09/13 10:32:43 d2.utils.events]: eta: 0:45:43 iter: 19 total_loss: 0.003062 loss_cls: 0.001598 loss_box_reg: 0 loss_keypoint: 0 loss_rpn_cls: 0.0005277 loss_rpn_loc: 0 time: 1.3875 data_time: 0.0270 lr: 4.9953e-06 max_mem: 5947M [09/13 10:33:10 d2.utils.events]: eta: 0:45:43 iter: 39 total_loss: 0.002124 loss_cls: 0.0005451 loss_box_reg: 0 loss_keypoint: 0 loss_rpn_cls: 0.001126 loss_rpn_loc: 0 time: 1.3738 data_time: 0.0046 lr: 9.9902e-06 max_mem: 5947M [09/13 10:33:21 d2.engine.hooks]: Overall training speed: 45 iterations in 0:01:03 (1.4115 s / it) [09/13 10:33:21 d2.engine.hooks]: Total training time: 0:01:03 (0:00:00 on hooks) [09/13 10:33:21 d2.utils.events]: eta: 0:45:36 iter: 47 total_loss: 0.001595 loss_cls: 0.0004478 loss_box_reg: 0 loss_keypoint: 0 loss_rpn_cls: 0.0008172 loss_rpn_loc: 0 time: 1.3836 data_time: 0.0061 lr: 1.1738e-05 max_mem: 5947M

eaedk commented 3 years ago

I HAVE A DATASET WITH 4 KEYPOINTS

AarohiSingla commented 3 years ago

AShare your dataset. I will try to solve the issue

On Mon, Sep 13, 2021 at 5:07 PM Emmanuel KOUPOH @.***> wrote:

I HAVE A DATASET WITH 4 KEYPOINTS

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/AarohiSingla/Key-Point-Detection-on-Custom-Dataset/issues/1#issuecomment-918105708, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOJ7RWW4CGOD6RHAJ3W2BI3UBXO7LANCNFSM5D5PIAXQ .

eaedk commented 3 years ago

I can just share the annotation file, the images are privates, I have no authorization to share. is it ok ?

melodjia commented 3 years ago

AShare your dataset. I will try to solve the issue … On Mon, Sep 13, 2021 at 5:07 PM Emmanuel KOUPOH @.***> wrote: I HAVE A DATASET WITH 4 KEYPOINTS — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOJ7RWW4CGOD6RHAJ3W2BI3UBXO7LANCNFSM5D5PIAXQ .

AarohiSingla, You say in the video on YouTube that you can use various tools for annotation, but do not say what you specifically used. Could you tell me which tool you are using for detecting keypoints? thanks a lot

eaedk commented 3 years ago

Hello, for annotation I used coco-annotator. please could you share your dataset with me and tell me what tool you used to annotate images.

AarohiSingla commented 3 years ago

I have used

https://sense.sixgill.com/

On Fri, 17 Sep 2021 at 3:10 AM, Emmanuel KOUPOH @.***> wrote:

Hello, for annotation I used coco-annotator. please could you share your dataset with me and tell me what tool you used to annotate images.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/AarohiSingla/Key-Point-Detection-on-Custom-Dataset/issues/1#issuecomment-921274513, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOJ7RWVOIZEOCGYWCE7FQWTUCJP5TANCNFSM5D5PIAXQ .

eaedk commented 3 years ago

please could you share a notebook with the training output, I would like to compare mine (I got some loss scores but the keypoint loss score stay at NaN)

eaedk commented 3 years ago

or just the output as I did above...

AarohiSingla commented 3 years ago

Hi, My keypoint loss value for this particular example is almost around the total loss value. Eg: if my total loss value is 0.9 then my keypoint loss value is 0.8 or 0.7

On Fri, Sep 17, 2021 at 3:01 PM Emmanuel KOUPOH @.***> wrote:

or just the output as I did above...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/AarohiSingla/Key-Point-Detection-on-Custom-Dataset/issues/1#issuecomment-921652750, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOJ7RWS67BJF43AQIYIGKXTUCMDILANCNFSM5D5PIAXQ .

eaedk commented 3 years ago

ok, there is a problem with my annotations, the points are set but the bounding box coordinates are all near to 0.

I used another dataset, but I have an issue the model_final.pth is not saved. grrrrrrrh you set in your notebook there are this : 

cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1  # hand
cfg.MODEL.RETINANET.NUM_CLASSES = 1

why the second line ??

eaedk commented 3 years ago

with the other dataset I got this as last output:

[09/17 16:18:39 d2.utils.events]: eta: 3 days, 4:08:46 iter: 6859 total_loss: 5.524 loss_cls: 0.5272 loss_box_reg: 0.4919 loss_keypoint: 4.274 loss_rpn_cls: 0.02668 loss_rpn_loc: 0.237 time: 1.4240 data_time: 0.0136 lr: 2.5e-05 max_mem: 4874M

Do you have any advice to help me improve please ?