AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )
http://pjreddie.com/darknet/
Other
21.75k stars 7.96k forks source link

Implement TridentNet with Yolo-head ~+2.2 AP@[.5, .95] #3363

Closed AlexeyAB closed 3 years ago

AlexeyAB commented 5 years ago

Implement TridentNet with Yolo-head

image


image


trident_block_details (1)


image

1_aq0q7gCvuNUqnMHh4cpnIw

AlexeyAB commented 5 years ago

Implemented!

Example:


59539364-a22d5880-8f05-11e9-8df5-0160a39f62e7

primepake commented 5 years ago

Why do you use resnet152 instead of resnet101 ? can you explaned it. tks

AlexeyAB commented 5 years ago

@hitle451997 Just to try. But yes, it seems the optimal is to use resnet 101 as it is done in the paper.

beHappy666 commented 5 years ago

@AlexeyAB Could you tell me how to change the last conv's filter number before yolo layer?

AlexeyAB commented 5 years ago

@beHappy666

filters=(classes+5)*4

deimsdeutsch commented 5 years ago

@AlexeyAB Can you please do a comparison with Yolov3-416 vs TridentNet-Resnet152.

GPU Memory Usage Inference Time mAP on mscoco Flops

Is TridentNet-Resnet152 stable ?

Thanks.

LukeAI commented 5 years ago

@deimsdeutsch https://github.com/AlexeyAB/darknet/issues/3114#issuecomment-494148968

keko950 commented 5 years ago

@beHappy666

filters=(classes+5)*4

Is this right? Im receiving Error: l.outputs == params.inputs filters= in the [convolutional]-layer doesn't correspond to classes= or mask= in [yolo]-layer

hvudeshi commented 5 years ago

Firstly, thank you so much for implementing TridentNet in darknet @AlexeyAB. But I have one doubt that as mentioned in TridentNet paper, only one branch is approximated during inference, so @AlexeyAB I want to know where exactly that is implemented in your darknet code?

AlexeyAB commented 5 years ago

@hardik22317

But I have one doubt that as mentioned in TridentNet paper, only one branch is approximated during inference,

Can you provide quote?

What do you mean?

hvudeshi commented 5 years ago

Hello @AlexeyAB,

Trident-2 trident-1 I am talking about the speed-up of TridentNet by using one major branch during inference. I have attached the images of the paper where the author quote about this process.

AlexeyAB commented 5 years ago

@hardik22317 No, Training 3-branches and Detection on 1-branch isn't implemented in my repo.

hvudeshi commented 5 years ago

@AlexeyAB how have you decided the number of trident blocks? Is there any specific reason behind keeping 6 out of 36 blocks of conv4 for Trident? In the paper, author has shown the best results by keeping 15 blocks out of 23 blocks of conv4 of resnet101 for trident. trident

hahakid commented 5 years ago

Implemented!

Example:

59539364-a22d5880-8f05-11e9-8df5-0160a39f62e7

I used the cfg file on kitti (3 classes), only modified the #class and last layers filters before yolo, at the end of training, following errors occured:

Error in `./darknet': free(): invalid pointer: 0x0000000000f893e0

And, at the end of testing, I got:

Error in `./darknet': double free or corruption (!prev): 0x0000000001341420 ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f2fd15c97e5] /lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7f2fd15d237a] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f2fd15d653c]

the full errors during training: Saving weights to /media/kid/space2/kittimodel/resnet152_trident_last.weights Saving weights to /media/kid/space2/kittimodel/resnet152_trident_final.weights Error in `./darknet': free(): invalid pointer: 0x0000000000f893e0 ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f610b4457e5] /lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7f610b44e37a] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f610b45253c] ./darknet[0x49851a] ./darknet[0x473681] ./darknet[0x496225] ./darknet[0x497bb8] ./darknet[0x406e90] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f610b3ee830] ./darknet[0x4087e9] ======= Memory map: ======== 00400000-00809000 r-xp 00000000 08:12 4474073 /home/kid/workspace/darknet/darknet 00a09000-00a0a000 r--p 00409000 08:12 4474073 /home/kid/workspace/darknet/darknet 00a0a000-00a0b000 rw-p 0040a000 08:12 4474073 /home/kid/workspace/darknet/darknet 00a0b000-00a0c000 rw-p 00000000 00:00 0 00c46000-f4c25000 rw-p 00000000 00:00 0 [heap] 200000000-200200000 rw-s 00000000 00:06 543 /dev/nvidiactl 200200000-200400000 ---p 00000000 00:00 0 200400000-200600000 rw-s 00000000 00:06 543 /dev/nvidiactl 200600000-202600000 rw-s 00000000 00:06 543 /dev/nvidiactl 202600000-205600000 rw-s 00000000 00:06 543 /dev/nvidiactl 205600000-206200000 ---p 00000000 00:00 0 206200000-206400000 rw-s 00000000 00:06 543 /dev/nvidiactl 206400000-206600000 rw-s 00000000 00:06 543 /dev/nvidiactl 206600000-206800000 rw-s 206600000 00:06 540 /dev/nvidia-uvm 206800000-206a00000 ---p 00000000 00:00 0 206a00000-206c00000 rw-s 00000000 00:06 543 /dev/nvidiactl 206c00000-300200000 ---p 00000000 00:00 0 400000000-400200000 rw-s 00000000 00:06 543 /dev/nvidiactl 400200000-400400000 ---p 00000000 00:00 0 400400000-400600000 rw-s 00000000 00:06 543 /dev/nvidiactl 400600000-402600000 rw-s 00000000 00:06 543 /dev/nvidiactl 402600000-405600000 rw-s 00000000 00:06 543 /dev/nvidiactl 405600000-406200000 ---p 00000000 00:00 0 406200000-406400000 rw-s 00000000 00:06 543 /dev/nvidiactl 406400000-406600000 rw-s 00000000 00:06 543 /dev/nvidiactl 406600000-406800000 rw-s 406600000 00:06 540 /dev/nvidia-uvm 406800000-406a00000 ---p 00000000 00:00 0 406a00000-406c00000 rw-s 00000000 00:06 543 /dev/nvidiactl 406c00000-500200000 ---p 00000000 00:00 0 10000000000-10404200000 ---p 00000000 00:00 0 7f5ee4000000-7f5ee4021000 rw-p 00000000 00:00 0 7f5ee4021000-7f5ee8000000 ---p 00000000 00:00 0 7f5ee8000000-7f5ee8021000 rw-p 00000000 00:00 0 7f5ee8021000-7f5eec000000 ---p 00000000 00:00 0 7f5eec000000-7f5eec021000 rw-p 00000000 00:00 0 7f5eec021000-7f5ef0000000 ---p 00000000 00:00 0 7f5ef2000000-7f5ef4000000 ---p 00000000 00:00 0 7f5ef4000000-7f5ef4021000 rw-p 00000000 00:00 0 7f5ef4021000-7f5ef8000000 ---p 00000000 00:00 0 7f5ef8000000-7f5ef8021000 rw-p 00000000 00:00 0 7f5ef8021000-7f5efc000000 ---p 00000000 00:00 0 7f5efc000000-7f5efc021000 rw-p 00000000 00:00 0 7f5efc021000-7f5f00000000 ---p 00000000 00:00 0 7f5f017ff000-7f5f037ff000 rw-s 00000000 00:05 24969238 /SYSV00000000 (deleted) 7f5f037ff000-7f5f03800000 ---p 00000000 00:00 0 7f5f03800000-7f5f04000000 rw-p 00000000 00:00 0 7f5f04000000-7f5f04021000 rw-p 00000000 00:00 0 7f5f04021000-7f5f08000000 ---p 00000000 00:00 0 7f5f09ffc000-7f5f09ffd000 ---p 00000000 00:00 0 7f5f09ffd000-7f5f0a7fd000 rw-p 00000000 00:00 0 7f5f0a7fd000-7f5f0a7fe000 ---p 00000000 00:00 0 7f5f0a7fe000-7f5f0affe000 rw-p 00000000 00:00 0 7f5f0affe000-7f5f0afff000 ---p 00000000 00:00 0 7f5f0afff000-7f5f0b7ff000 rw-p 00000000 00:00 0 7f5f0b7ff000-7f5f0b800000 ---p 00000000 00:00 0 7f5f0b800000-7f5f0c000000 rw-p 00000000 00:00 0 7f5f0c000000-7f5f0f527000 rw-p 00000000 00:00 0 7f5f0f527000-7f5f10000000 ---p 00000000 00:00 0 7f5f10000000-7f5f10021000 rw-p 00000000 00:00 0 7f5f10021000-7f5f14000000 ---p 00000000 00:00 0 7f5f14000000-7f5f174c8000 rw-p 00000000 00:00 0 7f5f174c8000-7f5f18000000 ---p 00000000 00:00 0 7f5f18000000-7f5f1b1c2000 rw-p 00000000 00:00 0 7f5f1b1c2000-7f5f1c000000 ---p 00000000 00:00 0 7f5f1c000000-7f5f1c021000 rw-p 00000000 00:00 0 7f5f1c021000-7f5f20000000 ---p 00000000 00:00 0 7f5f20000000-7f5f28000000 ---p 00000000 00:00 0 7f5f28000000-7f5f28021000 rw-p 00000000 00:00 0 7f5f28021000-7f5f2c000000 ---p 00000000 00:00 0 7f5f2c7fd000-7f5f2c7fe000 ---p 00000000 00:00 0 7f5f2c7fe000-7f5f2cffe000 rw-p 00000000 00:00 0 7f5f2cffe000-7f5f2cfff000 ---p 00000000 00:00 0 7f5f2cfff000-7f5f2d7ff000 rw-p 00000000 00:00 0 7f5f2d7ff000-7f5f2d800000 ---p 00000000 00:00 0 7f5f2d800000-7f5f2e000000 rw-p 00000000 00:00 0 7f5f2e000000-7f5f30000000 ---p 00000000 00:00 0 7f5f30000000-7f5f30001000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f30001000-7f5f30002000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f30002000-7f5f30003000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f30003000-7f5f30004000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f30004000-7f5f30005000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f30005000-7f5f30006000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f30006000-7f5f30007000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f30007000-7f5f30008000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f30008000-7f5f30009000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f30009000-7f5f3000a000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f3000a000-7f5f3000b000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f3000b000-7f5f3000c000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f3000c000-7f5f3000d000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f3000d000-7f5f3000e000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f3000e000-7f5f3000f000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f3000f000-7f5f30010000 rw-s 00000000 00:06 544 /dev/nvidia0 7f5f30010000-7f5f40000000 ---p 00000000 00:00 0 7f5f407fd000-7f5f407fe000 ---p 00000000 00:00 0 7f5f407fe000-7f5f40ffe000 rw-p 00000000 00:00 0 7f5f40ffe000-7f5f40fff000 ---p 00000000 00:00 0 7f5f40fff000-7f5f417ff000 rw-p 00000000 00:00 0 7f5f417ff000-7f5f41800000 ---p 00000000 00:00 0 7f5f41800000-7f5f42000000 rw-p 00000000 00:00 0 7f5f42000000-7f5f42600000 ---p 00000000 00:00 0 7f5f42600000-7f5f42800000 rw-s 00000000 00:05 2622007 /dev/zero (deleted) 7f5f42800000-7f5f42a00000 rw-s 00000000 00:06 543 /dev/nvidiactl 7f5f42a00000-7f5f42c00000 rw-s 00000000 00:05 2622008 /dev/zero (deleted) 7f5f42c00000-7f5f42e00000 ---p 00000000 00:00 0 7f5f42e00000-7f5f43000000 rw-s 00000000 00:06 543 /dev/nvidiactl 7f5f43000000-7f5f43200000 ---p 00000000 00:00 0 7f5f43200000-7f5f43400000 rw-s 00000000 00:05 2622010 /dev/zero (deleted) 7f5f43400000-7f5f436d6000 rw-s 00000000 00:06 543 /dev/nvidiactl 7f5f436d6000-7f5f44000000 ---p 00000000 00:00 0 7f5f44000000-7f5f44021000 rw-p 00000000 00:00 0 7f5f44021000-7f5f48000000 ---p 00000000 00:00 0 7f5f487fd000-7f5f487fe000 ---p 00000000 00:00 0 7f5f487fe000-7f5f48ffe000 rw-p 00000000 00:00 0 7f5f48ffe000-7f5f48fff000 ---p 00000000 00:00 0 7f5f48fff000-7f5f497ff000 rw-p 00000000 00:00 0 7f5f497ff000-7f5f49800000 ---p 00000000 00:00 0 7f5f49800000-7f5f4a000000 rw-p 00000000 00:00 0 7f5f4a000000-7f5f50000000 ---p 00000000 00:00 0 7f5f50000000-7f5f50022000 rw-p 00000000 00:00 0 7f5f50022000-7f5f54000000 ---p 00000000 00:00 0 7f5f5429c000-7f5f5429d000 ---p 00000000 00:00 0 7f5f5429d000-7f5f54a9d000 rw-p 00000000 00:00 0 7f5f54a9d000-7f5f54a9e000 ---p 00000000 00:00 0 7f5f54a9e000-7f5f5529e000 rw-p 00000000 00:00 0 7f5f5529e000-7f5f5529f000 ---p 00000000 00:00 0 7f5f5529f000-7f5f55a9f000 rw-p 00000000 00:00 0 7f5f55a9f000-7f5f55aa0000 ---p 00000000 00:00 0 7f5f55aa0000-7f5f562a0000 rw-p 00000000 00:00 0 7f5f562a0000-7f5f57c77000 r-xp 00000000 08:12 7090089 /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.418.56 7f5f57c77000-7f5f57e76000 ---p 019d7000 08:12 7090089 /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.418.56 7f5f57e76000-7f5f57ff0000 rw-p 019d6000 08:12 7090089 /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.418.56 7f5f57ff0000-7f5f58000000 rw-p 00000000 00:00 0 7f5f58000000-7f5f58021000 rw-p 00000000 00:00 0 7f5f58021000-7f5f5c000000 ---p 00000000 00:00 0 7f5f5c000000-7f5f5c021000 rw-p 00000000 00:00 0 7f5f5c021000-7f5f60000000 ---p 00000000 00:00 0 7f5f60000000-7f5f60363000 rw-s 00000000 00:05 2623726 /dev/zero (deleted) 7f5f60363000-7f5f68000000 ---p 00000000 00:00 0 7f5f685ee000-7f5f685ef000 ---p 00000000 00:00 0 7f5f685ef000-7f5f6a000000 rw-p 00000000 00:00 0 7f5f6a000000-7f5fa6000000 ---p 00000000 00:00 0 7f5faa000000-7f5fac000000 ---p 00000000 00:00 0 7f5fb2000000-7f5fb6000000 ---p 00000000 00:00 0 7f5fbc000000-7f5fbe000000 ---p 00000000 00:00 0 7f5fe4000000-7f5fec000000 ---p 00000000 00:00 0 7f5fee000000-7f5ff4000000 ---p 00000000 00:00 0 7f5ff6000000-7f5ff8000000 ---p 00000000 00:00 0 7f5ffa000000-7f6000000000 ---p 00000000 00:00 0 7f6004000000-7f600a000000 ---p 00000000 00:00 0 7f6054000000-7f6056000000 ---p 00000000 00:00 0 7f6058000000-7f605a000000 ---p 00000000 00:00 0 7f606a000000-7f606c000000 ---p 00000000 00:00 0 7f607c000000-7f607e000000 ---p 00000000 00:00 0 7f60845c6000-7f60847c6000 rw-s 00000000 00:05 2622009 /dev/zero (deleted) 7f60847c6000-7f60847cc000 r-xp 00000000 08:12 1969028 /usr/local/cuda-10.0/lib64/libOpenCL.so.1.1 7f60847cc000-7f60849cb000 ---p 00006000 08:12 1969028 /usr/local/cuda-10.0/lib64/libOpenCL.so.1.1 7f60849cb000-7f60849cd000 rw-p 00005000 08:12 1969028 /usr/local/cuda-10.0/lib64/libOpenCL.so.1.1 7f60849cd000-7f6085bde000 rw-p 00000000 00:00 0 7f60901cc000-7f60901cd000 ---p 00000000 00:00 0 7f60901cd000-7f60909cd000 rw-p 00000000 00:00 0 7f60981cc000-7f60981cd000 ---p 00000000 00:00 0 7f60981cd000-7f60989cd000 rw-p 00000000 00:00 0 7f60a41cc000-7f60a41cd000 ---p 00000000 00:00 0 7f60a41cd000-7f60a49cd000 rw-p 00000000 00:00 0 7f60ac198000-7f60ac1a3000 r-xp 00000000 08:12 7343247 /usr/lib/x86_64-linux-gnu/gio/modules/libdconfsettings.so 7f60ac1a3000-7f60ac3a3000 ---p 0000b000 08:12 7343247 /usr/lib/x86_64-linux-gnu/gio/modules/libdconfsettings.so 7f60ac3a3000-7f60ac3a4000 r--p 0000b000 08:12 7343247 /usr/lib/x86_64-linux-gnu/gio/modules/libdconfsettings.so 7f60ac3a4000-7f60ac3a5000 rw-p 0000c000 08:12 7343247 /usr/lib/x86_64-linux-gnu/gio/modules/libdconfsettings.so 7f60ac3a5000-7f60ac3ae000 r-xp 00000000 08:12 7087026 /usr/lib/x86_64-linux-gnu/libltdl.so.7.3.1 7f60ac3ae000-7f60ac5ad000 ---p 00009000 08:12 7087026 /usr/lib/x86_64-linux-gnu/libltdl.so.7.3.1 7f60ac5ad000-7f60ac5ae000 r--p 00008000 08:12 7087026 /usr/lib/x86_64-linux-gnu/libltdl.so.7.3.1 7f60ac5ae000-7f60ac5af000 rw-p 00009000 08:12 7087026 /usr/lib/x86_64-linux-gnu/libltdl.so.7.3.1 7f60ac5af000-7f60ac5c3000 r-xp 00000000 08:12 7087418 /usr/lib/x86_64-linux-gnu/libtdb.so.1.3.8 7f60ac5c3000-7f60ac7c2000 ---p 00014000 08:12 7087418 /usr/lib/x86_64-linux-gnu/libtdb.so.1.3.8 7f60ac7c2000-7f60ac7c3000 r--p 00013000 08:12 7087418 /usr/lib/x86_64-linux-gnu/libtdb.so.1.3.8 7f60ac7c3000-7f60ac7c4000 rw-p 00014000 08:12 7087418 /usr/lib/x86_64-linux-gnu/libtdb.so.1.3.8 7f60ac7c4000-7f60ac7cb000 r-xp 00000000 08:12 7087533 /usr/lib/x86_64-linux-gnu/libvorbisfile.so.3.3.7 7f60ac7cb000-7f60ac9cb000 ---p 00007000 08:12 7087533 /usr/lib/x86_64-linux-gnu/libvorbisfile.so.3.3.7 7f60ac9cb000-7f60ac9cc000 r--p 00007000 08:12 7087533 /usr/lib/x86_64-linux-gnu/libvorbisfile.so.3.3.7 7f60ac9cc000-7f60ac9cd000 rw-p 00008000 08:12 7087533 /usr/lib/x86_64-linux-gnu/libvorbisfile.so.3.3.7 7f60b2189000-7f60b2199000 r-xp 00000000 08:12 7086407 /usr/lib/x86_64-linux-gnu/libcanberra.so.0.2.5 7f60b2199000-7f60b2398000 ---p 00010000 08:12 7086407 /usr/lib/x86_64-linux-gnu/libcanberra.so.0.2.5 7f60b2398000-7f60b2399000 r--p 0000f000 08:12 7086407 /usr/lib/x86_64-linux-gnu/libcanberra.so.0.2.5 7f60b2399000-7f60b239a000 rw-p 00010000 08:12 7086407 /usr/lib/x86_64-linux-gnu/libcanberra.so.0.2.5 7f60b239a000-7f60b239e000 r-xp 00000000 08:12 7086403 /usr/lib/x86_64-linux-gnu/libcanberra-gtk.so.0.1.9 7f60b239e000-7f60b259d000 ---p 00004000 08:12 7086403 /usr/lib/x86_64-linux-gnu/libcanberra-gtk.so.0.1.9 7f60b259d000-7f60b259e000 r--p 00003000 08:12 7086403 /usr/lib/x86_64-linux-gnu/libcanberra-gtk.so.0.1.9 7f60b259e000-7f60b259f000 rw-p 00004000 08:12 7086403 /usr/lib/x86_64-linux-gnu/libcanberra-gtk.so.0.1.9 7f60b259f000-7f60b25a4000 r-xp 00000000 08:12 7343466 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libcanberra-gtk-module.so 7f60b25a4000-7f60b27a3000 ---p 00005000 08:12 7343466 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libcanberra-gtk-module.so 7f60b27a3000-7f60b27a4000 r--p 00004000 08:12 7343466 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libcanberra-gtk-module.so 7f60b27a4000-7f60b27a5000 rw-p 00005000 08:12 7343466 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libcanberra-gtk-module.so 7f60b27a5000-7f60b27d5000 r-xp 00000000 08:12 7343448 /usr/lib/x86_64-linux-gnu/gtk-2.0/2.10.0/engines/libmurrine.so 7f60b27d5000-7f60b29d5000 ---p 00030000 08:12 7343448 /usr/lib/x86_64-linux-gnu/gtk-2.0/2.10.0/engines/libmurrine.so 7f60b29d5000-7f60b29d6000 r--p 00030000 08:12 7343448 /usr/lib/x86_64-linux-gnu/gtk-2.0/2.10.0/engines/libmurrine.so 7f60b29d6000-7f60b29d7000 rw-p 00031000 08:12 7343448 /usr/lib/x86_64-linux-gnu/gtk-2.0/2.10.0/engines/libmurrine.so 7f60b29d7000-7f60b29e7000 r-xp 00000000 08:12 7087476 /usr/lib/x86_64-linux-gnu/libunity-gtk2-parser.so.0.0.0 7f60b29e7000-7f60b2be7000 ---p 00010000 08:12 7087476 /usr/lib/x86_64-linux-gnu/libunity-gtk2-parser.so.0.0.0 7f60b2be7000-7f60b2be8000 r--p 00010000 08:12 7087476 /usr/lib/x86_64-linux-gnu/libunity-gtk2-parser.so.0.0.0 7f60b2be8000-7f60b2be9000 rw-p 00011000 08:12 7087476 /usr/lib/x86_64-linux-gnu/libunity-gtk2-parser.so.0.0.0 7f60b2be9000-7f60b2bee000 r-xp 00000000 08:12 7343470 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libunity-gtk-module.so 7f60b2bee000-7f60b2ded000 ---p 00005000 08:12 7343470 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libunity-gtk-module.so 7f60b2ded000-7f60b2dee000 r--p 00004000 08:12 7343470 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libunity-gtk-module.so 7f60b2dee000-7f60b2def000 rw-p 00005000 08:12 7343470 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libunity-gtk-module.so 7f60b4000000-7f60bb200000 ---p 00000000 00:00 0 7f60bb200000-7f60bb400000 rw-s 00000000 00:05 2616908 /dev/zero (deleted) 7f60bb400000-7f60bb600000 ---p 00000000 00:00 0 7f60bb600000-7f60bb800000 rw-s 00000000 00:05 2616909 /dev/zero (deleted) 7f60bb800000-7f60bc000000 ---p 00000000 00:00 0 7f60c2000000-7f60cc000000 ---p 00000000 00:00 0 7f60cc000000-7f60cc021000 rw-p 00000000 00:00 0 7f60cc021000-7f60d0000000 ---p 00000000 00:00 0 7f60d00eb000-7f60d0135000 r-xp 00000000 08:12 397954 /lib/x86_64-linux-gnu/libdbus-1.so.3.14.6 7f60d0135000-7f60d0335000 ---p 0004a000 08:12 397954 /lib/x86_64-linux-gnu/libdbus-1.so.3.14.6 7f60d0335000-7f60d0336000 r--p 0004a000 08:12 397954 /lib/x86_64-linux-gnu/libdbus-1.so.3.14.6 7f60d0336000-7f60d0337000 rw-p 0004b000 08:12 397954 /lib/x86_64-linux-gnu/libdbus-1.so.3.14.6 7f60d0337000-7f60d0363000 r-xp 00000000 08:12 7086346 /usr/lib/x86_64-linux-gnu/libatspi.so.0.0.1 7f60d0363000-7f60d0562000 ---p 0002c000 08:12 7086346 /usr/lib/x86_64-linux-gnu/libatspi.so.0.0.1 7f60d0562000-7f60d0565000 r--p 0002b000 08:12 7086346 /usr/lib/x86_64-linux-gnu/libatspi.so.0.0.1 7f60d0565000-7f60d0566000 rw-p 0002e000 08:12 7086346 /usr/lib/x86_64-linux-gnu/libatspi.so.0.0.1 7f60d0566000-7f60d0593000 r-xp 00000000 08:12 7086340 /usr/lib/x86_64-linux-gnu/libatk-bridge-2.0.so.0.0.0 7f60d0593000-7f60d0792000 ---p 0002d000 08:12 7086340 /usr/lib/x86_64-linux-gnu/libatk-bridge-2.0.so.0.0.0 7f60d0792000-7f60d0793000 r--p 0002c000 08:12 7086340 /usr/lib/x86_64-linux-gnu/libatk-bridge-2.0.so.0.0.0 7f60d0793000-7f60d0795000 rw-p 0002d000 08:12 7086340 /usr/lib/x86_64-linux-gnu/libatk-bridge-2.0.so.0.0.0 7f60d0795000-7f60d0796000 r-xp 00000000 08:12 7343465 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libatk-bridge.so 7f60d0796000-7f60d0995000 ---p 00001000 08:12 7343465 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libatk-bridge.so 7f60d0995000-7f60d0996000 r--p 00000000 08:12 7343465 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libatk-bridge.so 7f60d0996000-7f60d0997000 rw-p 00001000 08:12 7343465 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libatk-bridge.so 7f60d0997000-7f60d099e000 r-xp 00000000 08:12 7086681 /usr/lib/x86_64-linux-gnu/libgailutil.so.18.0.1 7f60d099e000-7f60d0b9d000 ---p 00007000 08:12 7086681 /usr/lib/x86_64-linux-gnu/libgailutil.so.18.0.1 7f60d0b9d000-7f60d0b9e000 r--p 00006000 08:12 7086681 /usr/lib/x86_64-linux-gnu/libgailutil.so.18.0.1 7f60d0b9e000-7f60d0b9f000 rw-p 00007000 08:12 7086681 /usr/lib/x86_64-linux-gnu/libgailutil.so.18.0.1 7f60d0b9f000-7f60d0bed000 r-xp 00000000 08:12 7343468 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libgail.so 7f60d0bed000-7f60d0dec000 ---p 0004e000 08:12 7343468 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libgail.so 7f60d0dec000-7f60d0ded000 r--p 0004d000 08:12 7343468 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libgail.so 7f60d0ded000-7f60d0def000 rw-p 0004e000 08:12 7343468 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libgail.so 7f60d2000000-7f60d2400000 ---p 00000000 00:00 0 7f60d2400000-7f60d2600000 rw-s 00000000 00:05 2616903 /dev/zero (deleted) 7f60d2600000-7f60d2800000 rw-s 00000000 00:06 543 /dev/nvidiactl 7f60d2800000-7f60d2a00000 rw-s 00000000 00:05 2616904 /dev/zero (deleted) 7f60d2a00000-7f60d2c00000 ---p 00000000 00:00 0 7f60d2c00000-7f60d2e00000 rw-s 00000000 00:06 543 /dev/nvidiactl 7f60d2e00000-7f60d3000000 ---p 00000000 00:00 0 7f60d3000000-7f60d3200000 rw-s 00000000 00:05 2616906 /dev/zero (deleted) 7f60d3200000-7f60d34d6000 rw-s 00000000 00:06 543 /dev/nvidiactl 7f60d34d6000-7f60d4000000 ---p 00000000 00:00 0 7f60d4000000-7f60d4021000 rw-p 00000000 00:00 0 7f60d4021000-7f60d8000000 ---p 00000000 00:00 0 7f60d81da000-7f60d8b99000 r--p 00000000 08:12 7085227 /usr/lib/locale/locale-archive 7f60dafbb000-7f60dafbc000 ---p 00000000 00:00 0 7f60dafbc000-7f60db7bc000 rw-p 00000000 00:00 0 7f60e0000000-7f60e0001000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e0001000-7f60e0002000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e0002000-7f60e0003000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e0003000-7f60e0004000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e0004000-7f60e0005000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e0005000-7f60e0006000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e0006000-7f60e0007000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e0007000-7f60e0008000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e0008000-7f60e0009000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e0009000-7f60e000a000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e000a000-7f60e000b000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e000b000-7f60e000c000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e000c000-7f60e000d000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e000d000-7f60e000e000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e000e000-7f60e000f000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e000f000-7f60e0010000 rw-s 00000000 00:06 544 /dev/nvidia0 7f60e0010000-7f60f0000000 ---p 00000000 00:00 0 7f60f00ac000-7f60f00f2000 r--p 00000000 08:12 7475425 /usr/share/glib-2.0/schemas/gschemas.compiled 7f60f00f2000-7f60f0172000 r-xp 00000000 08:12 397987 /lib/x86_64-linux-gnu/libsystemd.so.0.14.0 7f60f0172000-7f60f0175000 r--p 0007f000 08:12 397987 /lib/x86_64-linux-gnu/libsystemd.so.0.14.0 7f60f0175000-7f60f0176000 rw-p 00082000 08:12 397987 /lib/x86_64-linux-gnu/libsystemd.so.0.14.0 7f60f0176000-7f60f0177000 rw-p 00000000 00:00 0 7f60f018c000-7f60f018d000 rw-p 00000000 00:00 0 7f60f018d000-7f60f018e000 rw-s 00000000 00:06 543 /dev/nvidiactl 7f60f018e000-7f60f018f000 rw-s 00000000 00:06 543 /dev/nvidiactl 7f60f018f000-7f60f0190000 rw-s 00000000 00:06 543 /dev/nvidiactl 7f60f0190000-7f60f0191000 rw-s 00000000 00:06 543 /dev/nvidiactl 7f60f0191000-7f60f0192000 rw-s 00000000 00:06 543 /dev/nvidiactl 7f60f0192000-7f60f0193000 rw-s 00000000 00:06 543 /dev/nvidiactl 7f60f0193000-7f60f0194000 rw-s 00000000 00:06 543 /dev/nvidiactl 7f60f0194000-7f60f01a5000 r-xp 00000000 08:12 7343469 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/liboverlay-scrollbar.so 7f60f01a5000-7f60f03a5000 ---p 00011000 08:12 7343469 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/liboverlay-scrollbar.so 7f60f03a5000-7f60f03a6000 r--p 00011000 08:12 7343469 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/liboverlay-scrollbar.so 7f60f03a6000-7f60f03a7000 rw-p 00012000 08:12 7343469 /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/liboverlay-scrollbar.so 7f60f03a7000-7f60f043e000 rw-p 00000000 00:00 0 7f60f043e000-7f60f063e000 rw-s 00000000 00:05 2616905 /dev/zero (deleted) 7f60f063e000-7f60f0740000 rw-p 00000000 00:00 0 7f60f0740000-7f60f0741000 ---p 00000000 00:00 0 7f60f0741000-7f60f0f41000 rw-p 00000000 00:00 0 7f60f0f41000-7f60f0f42000 ---p 00000000 00:00 0 7f60f0f42000-7f60f1742000 rw-p 00000000 00:00 0 7f60f1742000-7f60f7742000 ---p 00000000 00:00 0 7f60f7742000-7f60f7743000 ---p 00000000 00:00 0 7f60f7743000-7f60f7f43000 rw-p 00000000 00:00 0 7f60f7f43000-7f60f97f9000 r-xp 00000000 08:12 7086926 /usr/lib/x86_64-linux-gnu/libicudata.so.55.1 7f60f97f9000-7f60f99f8000 ---p 018b6000 08:12 7086926 /usr/lib/x86_64-linux-gnu/libicudata.so.55.1 7f60f99f8000-7f60f99f9000 r--p 018b5000 08:12 7086926 /usr/lib/x86_64-linux-gnu/libicudata.so.55.1 7f60f99f9000-7f60f99fa000 rw-p 018b6000 08:12 7086926 /usr/lib/x86_64-linux-gnu/libicudata.so.55.1 7f60f99fa000-7f60f99fd000 r-xp 00000000 08:12 398018 /lib/x86_64-linux-gnu/libkeyutils.so.1.5 7f60f99fd000-7f60f9bfc000 ---p 00003000 08:12 398018 /lib/x86_64-linux-gnu/libkeyutils.so.1.5 7f60f9bfc000-7f60f9bfd000 r--p 00002000 08:12 398018 /lib/x86_64-linux-gnu/libkeyutils.so.1.5 7f60f9bfd000-7f60f9bfe000 rw-p 00003000 08:12 398018 /lib/x86_64-linux-gnu/libkeyutils.so.1.5 7f60f9bfe000-7f60f9d7d000 r-xp 00000000 08:12 7086940 /usr/lib/x86_64-linux-gnu/libicuuc.so.55.1 7f60f9d7d000-7f60f9f7d000 ---p 0017f000 08:12 7086940 /usr/lib/x86_64-linux-gnu/libicuuc.so.55.1 7f60f9f7d000-7f60f9f8d000 r--p 0017f000 08:12 7086940 /usr/lib/x86_64-linux-gnu/libicuuc.so.55.1 7f60f9f8d000-7f60f9f8e000 rw-p 0018f000 08:12 7086940 /usr/lib/x86_64-linux-gnu/libicuuc.so.55.1 7f60f9f8e000-7f60f9f92000 rw-p 00000000 00:00 0 7f60f9f92000-7f60f9f9c000 r-xp 00000000 08:12 7086998 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 7f60f9f9c000-7f60fa19b000 ---p 0000a000 08:12 7086998 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 7f60fa19b000-7f60fa19c000 r--p 00009000 08:12 7086998 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 7f60fa19c000-7f60fa19d000 rw-p 0000a000 08:12 7086998 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 7f60fa19d000-7f60fa1a0000 r-xp 00000000 08:12 397966 /lib/x86_64-linux-gnu/libcom_err.so.2.1 7f60fa1a0000-7f60fa39f000 ---p 00003000 08:12 397966 /lib/x86_64-linux-gnu/libcom_err.so.2.1 7f60fa39f000-7f60fa3a0000 r--p 00002000 08:12 397966 /lib/x86_64-linux-gnu/libcom_err.so.2.1 7f60fa3a0000-7f60fa3a1000 rw-p 00003000 08:12 397966 /lib/x86_64-linux-gnu/libcom_err.so.2.1 7f60fa3a1000-7f60fa3cd000 r-xp 00000000 08:12 7086988 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 7f60fa3cd000-7f60fa5cc000 ---p 0002c000 08:12 7086988 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 7f60fa5cc000-7f60fa5ce000 r--p 0002b000 08:12 7086988 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 7f60fa5ce000-7f60fa5cf000 rw-p 0002d000 08:12 7086988 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 7f60fa5cf000-7f60fa5d0000 rw-p 00000000 00:00 0 7f60fa5d0000-7f60fa693000 r-xp 00000000 08:12 7086996 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 7f60fa693000-7f60fa893000 ---p 000c3000 08:12 7086996 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 7f60fa893000-7f60fa8a0000 r--p 000c3000 08:12 7086996 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 7f60fa8a0000-7f60fa8a2000 rw-p 000d0000 08:12 7086996 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 7f60fa8a2000-7f60fa8b4000 r-xp 00000000 08:12 398001 /lib/x86_64-linux-gnu/libgpg-error.so.0.17.0 7f60fa8b4000-7f60faab4000 ---p 00012000 08:12 398001 /lib/x86_64-linux-gnu/libgpg-error.so.0.17.0 7f60faab4000-7f60faab5000 r--p 00012000 08:12 398001 /lib/x86_64-linux-gnu/libgpg-error.so.0.17.0 7f60faab5000-7f60faab6000 rw-p 00013000 08:12 398001 /lib/x86_64-linux-gnu/libgpg-error.so.0.17.0Aborted (core dumped)

AlexeyAB commented 5 years ago

@hahakid Hi,

What command do you use?

May be it is because this weights file was trained before fixing this bug: https://github.com/AlexeyAB/darknet/commit/6761e832cc01776f9471f52367405151522bbd3f So you should train your TridentNet model from scratch.

hahakid commented 5 years ago

@hahakid Hi,

What command do you use?

May be it is because this weights file was trained before fixing this bug: 6761e83 So you should train your TridentNet model from scratch. Thanks for your kind reply!

./darknet detector train data cfg

I use normal train cmd from the begining (under ubuntu), because I use a hourglass backbone modifed from cornernet-lite (Squeeze). I see you attached '-map' in the end (to show red line), I am not quite understand train from scratch? Do you mean from pretrain weights? I can confirm the backbone is alright, I just replaced the normal Yolo with the TridentNet tails. Any suggest? All training is using the least code (I just checked).

The bug fixing related to share_layer, do you mean the tails sharing the weight free triple times? should I turn it back in my local code?


The TridentNet cfg you offered also show the same error.

AlexeyAB commented 5 years ago

@hahakid Hi, I fixed several bugs for TridentNet. I re-trained the model, test it and updated results for resnet152_trident.cfg.txt - it works well. Download the latest version of Darknet.

hahakid commented 5 years ago

@AlexeyAB Thanks for your hard work, I will try it on local machine. And reply later. I tested on my local machine with my own backbone, it works well. I will try it on server.

beHappy666 commented 5 years ago

@keko950 (classes+5)*(anchor number in each layer)

hvudeshi commented 5 years ago

@AlexeyAB I have implemented TridentNet using resnet_152 as backbone on my custom data. I have used the cfg "resnet152_trident.cfg" that you have shared in your cfg folder. It worked. I also want to try it out on smaller network, hence I used Resnet_50 as my backbone with yolo head. But, I am facing the issue of avg loss going nan after 900 iterations on 4 GPUs. I have also tried on 1 GPU and in that case avg loss going nan after 1500 iterations.

I used 6 blocks of conv4 and 3 blocks of conv5 as Trident blocks.

I am not able to understand whether this problem is because of something wrong in training (cfg) or something specific to TridentNet using Resnet50 as backbone with yolo head. Below are my configuration parameters and I am also attaching cfg for your reference:- batch=64 subdivisions=64 height=608 width=608 channels=3 burn_in=1000 learning_rate=0.00025 max_batches=320000 policy=poly power=4 momentum=0.9 decay=0.0005

resnet50_trident.txt

Kindly help me solve this issue. Thank you.

AlexeyAB commented 5 years ago

@hardik22317 Use first lines from yolov3: https://github.com/AlexeyAB/darknet/blob/b0b1584a069c12a214791c339bbff9dc3904283e/cfg/yolov3.cfg#L1-L23

Also get pre-trained file: https://github.com/AlexeyAB/darknet/blob/b0b1584a069c12a214791c339bbff9dc3904283e/build/darknet/x64/partial.cmd#L50

And train by using this weights ./darknet detector train data/obj.data resnet50_trident.txt resnet50.65 -map

hvudeshi commented 5 years ago

@hardik22317 Use first lines from yolov3:

https://github.com/AlexeyAB/darknet/blob/b0b1584a069c12a214791c339bbff9dc3904283e/cfg/yolov3.cfg#L1-L23

Also get pre-trained file:

https://github.com/AlexeyAB/darknet/blob/b0b1584a069c12a214791c339bbff9dc3904283e/build/darknet/x64/partial.cmd#L50

And train by using this weights ./darknet detector train data/obj.data resnet50_trident.txt resnet50.65 -map

@AlexeyAB I tried that but after 5000 iterations, avg_loss became nan. Is this because I have used Resnet50? Should I try larger backbone like Resnet101? Could you please help me with this issue?

craterkamath commented 4 years ago

I am trying to train this for 4 classes, even after changing all the [yolo] and [convolutional] layers (before the yolo), the assertion fails. Can someone please help me with this.

CUDA-version: 10020 (10020), cuDNN: 7.6.5, GPU count: 4 OpenCV version: 3.4.3 Prepare additional network for mAP calculation... net.optimized_memory = 0 batch = 1, time_steps = 1, train = 0 layer filters size/strd(dil) input output 0 conv 64 7 x 7/ 2 544 x 544 x 3 -> 272 x 272 x 64 1.392 BF 1 max 2x 2/ 2 272 x 272 x 64 -> 136 x 136 x 64 0.005 BF 2 conv 64 1 x 1/ 1 136 x 136 x 64 -> 136 x 136 x 64 0.152 BF 3 conv 64 3 x 3/ 1 136 x 136 x 64 -> 136 x 136 x 64 1.364 BF 4 conv 256 1 x 1/ 1 136 x 136 x 64 -> 136 x 136 x 256 0.606 BF 5 Shortcut Layer: 1, wt = 0, wn = 0, outputs: 136 x 136 x 256 0.005 BF ( 136 x 136 x 256) + ( 136 x 136 x 64) 6 conv 64 1 x 1/ 1 136 x 136 x 256 -> 136 x 136 x 64 0.606 BF 7 conv 64 3 x 3/ 1 136 x 136 x 64 -> 136 x 136 x 64 1.364 BF 8 conv 256 1 x 1/ 1 136 x 136 x 64 -> 136 x 136 x 256 0.606 BF 9 Shortcut Layer: 5, wt = 0, wn = 0, outputs: 136 x 136 x 256 0.005 BF 10 conv 64 1 x 1/ 1 136 x 136 x 256 -> 136 x 136 x 64 0.606 BF 11 conv 64 3 x 3/ 1 136 x 136 x 64 -> 136 x 136 x 64 1.364 BF 12 conv 256 1 x 1/ 1 136 x 136 x 64 -> 136 x 136 x 256 0.606 BF 13 Shortcut Layer: 9, wt = 0, wn = 0, outputs: 136 x 136 x 256 0.005 BF 14 conv 128 1 x 1/ 1 136 x 136 x 256 -> 136 x 136 x 128 1.212 BF 15 conv 128 3 x 3/ 2 136 x 136 x 128 -> 68 x 68 x 128 1.364 BF 16 conv 512 1 x 1/ 1 68 x 68 x 128 -> 68 x 68 x 512 0.606 BF 17 Shortcut Layer: 13, wt = 0, wn = 0, outputs: 68 x 68 x 512 0.002 BF darknet: ./src/parser.c:880: parse_shortcut: Assertion `params.w == net.layers[index].out_w && params.h == net.layers[index].out_h' failed. Aborted (core dumped)

This is my cfg file: resnet152_trident.cfg.txt

AlexeyAB commented 4 years ago

@craterkamath I fixed resnet152_tridet.cfg try new cfg-file: https://github.com/AlexeyAB/darknet/commit/57264d13721e72b4af107e7504b7f351fffd5033

lucasjinreal commented 4 years ago

@AlexeyAB why not using trident in yolov4?

AlexeyAB commented 4 years ago

@jinfagang

Networks SAPD, RPDet, SM-NAS, ATSS, CenterMask, EfficientDet... are more optimal (AP/FPS) than TridentNet:

comparison_gpus