search

forresti / SqueezeNet

SqueezeNet: AlexNet-level accuracy with 50x fewer parameters
BSD 2-Clause "Simplified" License
2.17k stars 723 forks source link

Darknet/YOLO or Caffe/R-FCN + Squeezenet #15

Closed Grabber closed 7 years ago

Grabber commented 8 years ago

I think a very interesting combination with SqueezeNet is RFCN or YOLO for object detection. I'm trying to port SqueezeNet from Caffe to Darknet + YOLO.

Could someone help to review it?

It is a port from v1.1

squeezenet.cfg

[net]
batch=64
subdivisions=1
height=227
width=227
channels=3
momentum=0.9
decay=0.0005

learning_rate=0.001
policy=steps
steps=20,40,60,80,20000,30000
scales=5,5,2,2,.1,.1
max_batches=40000

[crop]
crop_width=227
crop_height=227
flip=0
angle=0
saturation = 1.5
exposure = 1.5

# SqueezeNet: conv1
[convolutional]
filters=64
size=3
stride=2
activation=relu

# SqueezeNet: pool1
[maxpool]
size=3
stride=2

# SqueezeNet: fire2/squeeze1x1
[convolutional]
filters=16
size=1
activation=relu

# SqueezeNet: fire2/expand1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire2/expand3x3
[convolutional]
filters=64
size=3
pad=1
activation=relu

# SqueezeNet: fire2/concat
[route]
layers=-3

# SqueezeNet: fire3/squeeze1x1
[convolutional]
filters=16
size=1
activation=relu

# SqueezeNet:fire3/expand1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire3/expand3x3
[convolutional]
filters=64
size=3
pad=1
activation=relu

# SqueezeNet: fire3/concat
[route]
layers=-3

# SqueezeNet: pool3
[maxpool]
size=3
stride=2

# SqueezeNet: fire4/squeeze1x1
[convolutional]
filters=32
size=1
activation=relu

# SqueezeNet: fire4/expand1x1
[convolutional]
filters=128
size=1
activation=relu

# SqueezeNet: fire4/expand3x3
[convolutional]
filters=128
size=3
pad=1
activation=relu

# SqueezeNet: fire4/concat
[route]
layers=-3

# SqueezeNet: fire5/squeeze1x1
[convolutional]
filters=32
size=1
activation=relu

# SqueezeNet: fire5/expand1x1
[convolutional]
filters=128
size=1
activation=relu

# SqueezeNet: fire5/expand3x3
[convolutional]
filters=128
size=3
pad=1
activation=relu

# SqueezeNet: fire5/concat
[route]
layers=-3

# SqueezeNet: pool5
[maxpool]
size=3
stride=2

# SqueezeNet: fire6/squeeze1x1
[convolutional]
filters=48
size=1
activation=relu

# SqueezeNet: fire6/expand1x1
[convolutional]
filters=192
size=1
activation=relu

# SqueezeNet: fire6/expand3x3
[convolutional]
filters=192
size=3
pad=1
activation=relu

# SqueezeNet: fire6/concat
[route]
layers=-3

# SqueezeNet: fire7/squeeze1x1
[convolutional]
filters=48
size=1
activation=relu

# SqueezeNet: fire7/expand1x1
[convolutional]
filters=192
size=1
activation=relu

# SqueezeNet: fire7/expand3x3
[convolutional]
filters=192
size=3
pad=1
activation=relu

# SqueezeNet: fire7/concat
[route]
layers=-3

# SqueezeNet: fire8/squeeze1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire8/expand1x1
[convolutional]
filters=256
size=1
activation=relu

# SqueezeNet: fire8/expand3x3
[convolutional]
filters=256
size=3
pad=1
activation=relu

# SqueezeNet: fire8/concat
[route]
layers=-3

# SqueezeNet: fire9/squeeze1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire9/expand1x1
[convolutional]
filters=256
size=1
activation=relu

# SqueezeNet: fire9/expand3x3
[convolutional]
filters=256
size=3
pad=1
activation=relu

# SqueezeNet: fire9/concat
[route]
layers=-3

# SqueezeNet: drop9
[dropout]
probability=.5

# SqueezeNet: conv10
[convolutional]
filters=1000
size=1
activation=relu

# SqueezeNet: pool10
[avgpool]

# YoLo: output = (5 * 2 + CLASSES) * SIDE^2
[connected]
output=784
activation=linear

# YoLo
[detection]
classes=1
coords=4
rescore=1
side=7
num=3
softmax=0
sqrt=1
jitter=.2

object_scale=1
noobject_scale=.5
class_scale=1
coord_scale=5

I'm no sure how to exactly port these cases:

    weight_filler {
      type: "xavier"
    }
    weight_filler {
      type: "gaussian"
      mean: 0.0
      std: 0.01
    }

Concat layers are strange too, don't know what index use on [route] frame=-?

forresti commented 8 years ago

Great idea! I'm definitely interested in this. Perhaps Mr Darknet (@pjreddie) may also care to weigh in.

The Darknet google group might also be a good place to ask this question: https://groups.google.com/forum/#!forum/darknet

I'm not that experienced with Darknet yet. Do we know if it can handle DAG-shaped CNNs, or only linked lists? I guess this might be the point of the "route" layer, but I haven't looked at darknet/src/route_layer.c in detail.

Grabber commented 8 years ago

We may could invite the R-FCN author too, what do you think @daijifeng001?

Grabber commented 8 years ago

@forresti I think Darknet implements a DAG style except there is no bi-directional sense of nodes connections. In other works, .cfg processing is procedural, but shortcut and route layers allow us to connect previous nodes/connections to the current one.

Grabber commented 8 years ago

I found some errors on my previous Darknet/squeezenet.cfg... still got stuck at 5~6 loss error.

batch=128
subdivisions=2
height=227
width=227
channels=3
momentum=0.9
decay=0.0005

learning_rate=0.001
policy=steps
steps=20,40,60,80,20000,30000
scales=5,5,2,2,.1,.1
max_batches=40000

[crop]
crop_width=227
crop_height=227
flip=0
angle=0
saturation = 1.5
exposure = 1.5

# SqueezeNet: conv1
[convolutional]
filters=64
size=3
stride=2
activation=relu

# SqueezeNet: pool1
[maxpool]
size=3
stride=2

# SqueezeNet: fire2/squeeze1x1
[convolutional]
filters=16
size=1
activation=relu

# SqueezeNet: fire2/expand1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire2/shortcut
[shortcut]
from=-2

# SqueezeNet: fire2/expand3x3
[convolutional]
filters=64
size=3
pad=1
activation=relu

# SqueezeNet: fire2/route
[route]
layers=-3,-1

# SqueezeNet: fire3/squeeze1x1
[convolutional]
filters=16
size=1
activation=relu

# SqueezeNet:fire3/expand1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire3/shortcut
[shortcut]
from=-2

# SqueezeNet: fire3/expand3x3
[convolutional]
filters=64
size=3
pad=1
activation=relu

# SqueezeNet: fire3/route
[route]
layers=-3,-1

# SqueezeNet: pool3
[maxpool]
size=3
stride=2

# SqueezeNet: fire4/squeeze1x1
[convolutional]
filters=32
size=1
activation=relu

# SqueezeNet: fire4/expand1x1
[convolutional]
filters=128
size=1
activation=relu

# SqueezeNet: fire4/shortcut
[shortcut]
from=-2

# SqueezeNet: fire4/expand3x3
[convolutional]
filters=128
size=3
pad=1
activation=relu

# SqueezeNet: fire4/route
[route]
layers=-3,-1

# SqueezeNet: fire5/squeeze1x1
[convolutional]
filters=32
size=1
activation=relu

# SqueezeNet: fire5/expand1x1
[convolutional]
filters=128
size=1
activation=relu

# SqueezeNet: fire5/shortcut
[shortcut]
from=-2

# SqueezeNet: fire5/expand3x3
[convolutional]
filters=128
size=3
pad=1
activation=relu

# SqueezeNet: fire5/route
[route]
layers=-3,-1

# SqueezeNet: pool5
[maxpool]
size=3
stride=2

# SqueezeNet: fire6/squeeze1x1
[convolutional]
filters=48
size=1
activation=relu

# SqueezeNet: fire6/expand1x1
[convolutional]
filters=192
size=1
activation=relu

# SqueezeNet: fire6/shortcut
[shortcut]
from=-2

# SqueezeNet: fire6/expand3x3
[convolutional]
filters=192
size=3
pad=1
activation=relu

# SqueezeNet: fire6/route
[route]
layers=-3,-1

# SqueezeNet: fire7/squeeze1x1
[convolutional]
filters=48
size=1
activation=relu

# SqueezeNet: fire7/expand1x1
[convolutional]
filters=192
size=1
activation=relu

# SqueezeNet: fire7/shortcut
[shortcut]
from=-2

# SqueezeNet: fire7/expand3x3
[convolutional]
filters=192
size=3
pad=1
activation=relu

# SqueezeNet: fire7/route
[route]
layers=-3,-1

# SqueezeNet: fire8/squeeze1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire8/expand1x1
[convolutional]
filters=256
size=1
activation=relu

# SqueezeNet: fire8/shortcut
[shortcut]
from=-2

# SqueezeNet: fire8/expand3x3
[convolutional]
filters=256
size=3
pad=1
activation=relu

# SqueezeNet: fire8/route
[route]
layers=-3,-1

# SqueezeNet: fire9/squeeze1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire9/expand1x1
[convolutional]
filters=256
size=1
activation=relu

# SqueezeNet: fire9/shortcut
[shortcut]
from=-2

# SqueezeNet: fire9/expand3x3
[convolutional]
filters=256
size=3
pad=1
activation=relu

# SqueezeNet: fire9/route
[route]
layers=-3,-1

# SqueezeNet: drop9
[dropout]
probability=.5

# SqueezeNet: conv10
[convolutional]
filters=1000
size=1
activation=relu

# SqueezeNet: pool10
[avgpool]

# YoLo: output = (5 * 2 + CLASSES) * SIDE^2
[connected]
output=539
activation=linear

# YoLo
[detection]
classes=1
coords=4
rescore=1
side=7
num=2
softmax=0
sqrt=1
jitter=.2

object_scale=1
noobject_scale=.5
class_scale=1
coord_scale=5```
ghost commented 8 years ago

Thank you for sharing. However, your model triggers a seg fault when running the yolo training. Have you changed the source code as well ? thx.

Grabber commented 8 years ago

@djebm2, I have not modified the source code to run it. By the way I have a newer version that is now training with loss errors below 1.

[net]
batch=128
subdivisions=2
height=227
width=227
channels=3
momentum=0.9
decay=0.0005

learning_rate=0.001
policy=steps
steps=20,40,60,80,20000,30000
scales=5,5,2,2,.1,.1
max_batches=40000

[crop]
crop_width=227
crop_height=227
flip=0
angle=0
saturation = 1.5
exposure = 1.5

# SqueezeNet: conv1
[convolutional]
filters=64
size=3
stride=2
activation=relu

# SqueezeNet: pool1
[maxpool]
size=3
stride=2

# SqueezeNet: fire2/squeeze1x1
[convolutional]
filters=16
size=1
activation=relu

# SqueezeNet: fire2/expand1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire2/shortcut
[route]
layers=-2

# SqueezeNet: fire2/expand3x3
[convolutional]
filters=64
size=3
pad=1
activation=relu

# SqueezeNet: fire2/route
[route]
layers=-3,-1

# SqueezeNet: fire3/squeeze1x1
[convolutional]
filters=16
size=1
activation=relu

# SqueezeNet:fire3/expand1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire3/shortcut
[route]
layers=-2

# SqueezeNet: fire3/expand3x3
[convolutional]
filters=64
size=3
pad=1
activation=relu

# SqueezeNet: fire3/route
[route]
layers=-3,-1

# SqueezeNet: pool3
[maxpool]
size=3
stride=2

# SqueezeNet: fire4/squeeze1x1
[convolutional]
filters=32
size=1
activation=relu

# SqueezeNet: fire4/expand1x1
[convolutional]
filters=128
size=1
activation=relu

# SqueezeNet: fire4/shortcut
[route]
layers=-2

# SqueezeNet: fire4/expand3x3
[convolutional]
filters=128
size=3
pad=1
activation=relu

# SqueezeNet: fire4/route
[route]
layers=-3,-1

# SqueezeNet: fire5/squeeze1x1
[convolutional]
filters=32
size=1
activation=relu

# SqueezeNet: fire5/expand1x1
[convolutional]
filters=128
size=1
activation=relu

# SqueezeNet: fire5/shortcut
[route]
layers=-2

# SqueezeNet: fire5/expand3x3
[convolutional]
filters=128
size=3
pad=1
activation=relu

# SqueezeNet: fire5/route
[route]
layers=-3,-1

# SqueezeNet: pool5
[maxpool]
size=3
stride=2

# SqueezeNet: fire6/squeeze1x1
[convolutional]
filters=48
size=1
activation=relu

# SqueezeNet: fire6/expand1x1
[convolutional]
filters=192
size=1
activation=relu

# SqueezeNet: fire6/shortcut
[route]
layers=-2

# SqueezeNet: fire6/expand3x3
[convolutional]
filters=192
size=3
pad=1
activation=relu

# SqueezeNet: fire6/route
[route]
layers=-3,-1

# SqueezeNet: fire7/squeeze1x1
[convolutional]
filters=48
size=1
activation=relu

# SqueezeNet: fire7/expand1x1
[convolutional]
filters=192
size=1
activation=relu

# SqueezeNet: fire7/shortcut
[route]
layers=-2

# SqueezeNet: fire7/expand3x3
[convolutional]
filters=192
size=3
pad=1
activation=relu

# SqueezeNet: fire7/route
[route]
layers=-3,-1

# SqueezeNet: fire8/squeeze1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire8/expand1x1
[convolutional]
filters=256
size=1
activation=relu

# SqueezeNet: fire8/shortcut
[route]
layers=-2

# SqueezeNet: fire8/expand3x3
[convolutional]
filters=256
size=3
pad=1
activation=relu

# SqueezeNet: fire8/route
[route]
layers=-3,-1

# SqueezeNet: fire9/squeeze1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire9/expand1x1
[convolutional]
filters=256
size=1
activation=relu

# SqueezeNet: fire9/shortcut
[route]
layers=-2

# SqueezeNet: fire9/expand3x3
[convolutional]
filters=256
size=3
pad=1
activation=relu

# SqueezeNet: fire9/route
[route]
layers=-3,-1

# SqueezeNet: drop9
[dropout]
probability=.5

# SqueezeNet: conv10
[convolutional]
filters=1000
size=1
activation=relu

# SqueezeNet: pool10
[avgpool]

# YoLo: output = (5 * 2 + CLASSES) * SIDE^2
[connected]
output=539
activation=linear

# YoLo
[detection]
classes=1
coords=4
rescore=1
side=7
num=2
softmax=0
sqrt=1
jitter=.2

object_scale=1
noobject_scale=.5
class_scale=1
coord_scale=5
pjreddie commented 8 years ago

Cfg file looks good, I think the first layer is 7x7 though right? Have you pre trained in image net? I'll try when I get home

On Sep 7, 2016 7:23 PM, "Luiz Vitor Martinez Cardoso" < notifications@github.com> wrote:

@djebm2 https://github.com/djebm2, I have not modified the source code to run it. By the way I have a newer version that is now training with loss errors below 1.

[net] batch=128 subdivisions=2 height=227 width=227 channels=3 momentum=0.9 decay=0.0005

learning_rate=0.001 policy=steps steps=20,40,60,80,20000,30000 scales=5,5,2,2,.1,.1 max_batches=40000

[crop] crop_width=227 crop_height=227 flip=0 angle=0 saturation = 1.5 exposure = 1.5

SqueezeNet: conv1

[convolutional] filters=64 size=3 stride=2 activation=relu

SqueezeNet: pool1

[maxpool] size=3 stride=2

SqueezeNet: fire2/squeeze1x1

[convolutional] filters=16 size=1 activation=relu

SqueezeNet: fire2/expand1x1

[convolutional] filters=64 size=1 activation=relu

SqueezeNet: fire2/shortcut

[route] layers=-2

SqueezeNet: fire2/expand3x3

[convolutional] filters=64 size=3 pad=1 activation=relu

SqueezeNet: fire2/route

[route] layers=-3,-1

SqueezeNet: fire3/squeeze1x1

[convolutional] filters=16 size=1 activation=relu

SqueezeNet:fire3/expand1x1

[convolutional] filters=64 size=1 activation=relu

SqueezeNet: fire3/shortcut

[route] layers=-2

SqueezeNet: fire3/expand3x3

[convolutional] filters=64 size=3 pad=1 activation=relu

SqueezeNet: fire3/route

[route] layers=-3,-1

SqueezeNet: pool3

[maxpool] size=3 stride=2

SqueezeNet: fire4/squeeze1x1

[convolutional] filters=32 size=1 activation=relu

SqueezeNet: fire4/expand1x1

[convolutional] filters=128 size=1 activation=relu

SqueezeNet: fire4/shortcut

[route] layers=-2

SqueezeNet: fire4/expand3x3

[convolutional] filters=128 size=3 pad=1 activation=relu

SqueezeNet: fire4/route

[route] layers=-3,-1

SqueezeNet: fire5/squeeze1x1

[convolutional] filters=32 size=1 activation=relu

SqueezeNet: fire5/expand1x1

[convolutional] filters=128 size=1 activation=relu

SqueezeNet: fire5/shortcut

[route] layers=-2

SqueezeNet: fire5/expand3x3

[convolutional] filters=128 size=3 pad=1 activation=relu

SqueezeNet: fire5/route

[route] layers=-3,-1

SqueezeNet: pool5

[maxpool] size=3 stride=2

SqueezeNet: fire6/squeeze1x1

[convolutional] filters=48 size=1 activation=relu

SqueezeNet: fire6/expand1x1

[convolutional] filters=192 size=1 activation=relu

SqueezeNet: fire6/shortcut

[route] layers=-2

SqueezeNet: fire6/expand3x3

[convolutional] filters=192 size=3 pad=1 activation=relu

SqueezeNet: fire6/route

[route] layers=-3,-1

SqueezeNet: fire7/squeeze1x1

[convolutional] filters=48 size=1 activation=relu

SqueezeNet: fire7/expand1x1

[convolutional] filters=192 size=1 activation=relu

SqueezeNet: fire7/shortcut

[route] layers=-2

SqueezeNet: fire7/expand3x3

[convolutional] filters=192 size=3 pad=1 activation=relu

SqueezeNet: fire7/route

[route] layers=-3,-1

SqueezeNet: fire8/squeeze1x1

[convolutional] filters=64 size=1 activation=relu

SqueezeNet: fire8/expand1x1

[convolutional] filters=256 size=1 activation=relu

SqueezeNet: fire8/shortcut

[route] layers=-2

SqueezeNet: fire8/expand3x3

[convolutional] filters=256 size=3 pad=1 activation=relu

SqueezeNet: fire8/route

[route] layers=-3,-1

SqueezeNet: fire9/squeeze1x1

[convolutional] filters=64 size=1 activation=relu

SqueezeNet: fire9/expand1x1

[convolutional] filters=256 size=1 activation=relu

SqueezeNet: fire9/shortcut

[route] layers=-2

SqueezeNet: fire9/expand3x3

[convolutional] filters=256 size=3 pad=1 activation=relu

SqueezeNet: fire9/route

[route] layers=-3,-1

SqueezeNet: drop9

[dropout] probability=.5

SqueezeNet: conv10

[convolutional] filters=1000 size=1 activation=relu

SqueezeNet: pool10

[avgpool]

YoLo: output = (5 * 2 + CLASSES) * SIDE^2

[connected] output=539 activation=linear

YoLo

[detection] classes=1 coords=4 rescore=1 side=7 num=2 softmax=0 sqrt=1 jitter=.2

object_scale=1 noobject_scale=.5 class_scale=1 coord_scale=5```

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DeepScale/SqueezeNet/issues/15#issuecomment-245475954, or mute the thread https://github.com/notifications/unsubscribe-auth/AAScHJOX2a7ZQVMLdWoIu42Oo4ZhHo4uks5qn3G5gaJpZM4JD4oZ .

Grabber commented 8 years ago

@pjreddie it is good to hear from you buddy!

Please check my latest versions of SqueezeNet for Darknet with route layers and batchnormalization. Please the check following variants:

[net]
batch=64
subdivisions=2
height=227
width=227
channels=3
momentum=0.9
decay=0.0005

learning_rate=0.001
policy=poly
power=1
max_batches=20000

[crop]
crop_width=227
crop_height=227
flip=0
angle=0
saturation=1.5
exposure=1.5

# SqueezeNet: conv1
[convolutional]
filters=64
size=3
stride=2
activation=relu

# SqueezeNet: pool1
[maxpool]
size=3
stride=2

[batchnorm]

# SqueezeNet: fire2/squeeze1x1
[convolutional]
filters=16
size=1
activation=relu

# SqueezeNet: fire2/expand1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire2/shortcut
[shortcut]
from=-2
activation=leaky

# SqueezeNet: fire2/expand3x3
[convolutional]
filters=64
size=3
pad=1
activation=relu

# SqueezeNet: fire2/route
[route]
layers=-3,-1

[batchnorm]

# SqueezeNet: fire3/squeeze1x1
[convolutional]
filters=16
size=1
activation=relu

# SqueezeNet:fire3/expand1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire3/shortcut
[shortcut]
from=-2
activation=leaky

# SqueezeNet: fire3/expand3x3
[convolutional]
filters=64
size=3
pad=1
activation=relu

# SqueezeNet: fire3/route
[route]
layers=-3,-1

[batchnorm]

# SqueezeNet: pool3
[maxpool]
size=3
stride=2

[batchnorm]

# SqueezeNet: fire4/squeeze1x1
[convolutional]
filters=32
size=1
activation=relu

# SqueezeNet: fire4/expand1x1
[convolutional]
filters=128
size=1
activation=relu

# SqueezeNet: fire4/shortcut
[shortcut]
from=-2
activation=leaky

# SqueezeNet: fire4/expand3x3
[convolutional]
filters=128
size=3
pad=1
activation=relu

# SqueezeNet: fire4/route
[route]
layers=-3,-1

[batchnorm]

# SqueezeNet: fire5/squeeze1x1
[convolutional]
filters=32
size=1
activation=relu

# SqueezeNet: fire5/expand1x1
[convolutional]
filters=128
size=1
activation=relu

# SqueezeNet: fire5/shortcut
[shortcut]
from=-2
activation=leaky

# SqueezeNet: fire5/expand3x3
[convolutional]
filters=128
size=3
pad=1
activation=relu

# SqueezeNet: fire5/route
[route]
layers=-3,-1

# SqueezeNet: pool5
[maxpool]
size=3
stride=2

[batchnorm]

# SqueezeNet: fire6/squeeze1x1
[convolutional]
filters=48
size=1
activation=relu

# SqueezeNet: fire6/expand1x1
[convolutional]
filters=192
size=1
activation=relu

# SqueezeNet: fire6/shortcut
[shortcut]
from=-2
activation=leaky

# SqueezeNet: fire6/expand3x3
[convolutional]
filters=192
size=3
pad=1
activation=relu

# SqueezeNet: fire6/route
[route]
layers=-3,-1

[batchnorm]

# SqueezeNet: fire7/squeeze1x1
[convolutional]
filters=48
size=1
activation=relu

# SqueezeNet: fire7/expand1x1
[convolutional]
filters=192
size=1
activation=relu

# SqueezeNet: fire7/shortcut
[shortcut]
from=-2
activation=leaky

# SqueezeNet: fire7/expand3x3
[convolutional]
filters=192
size=3
pad=1
activation=relu

# SqueezeNet: fire7/route
[route]
layers=-3,-1

[batchnorm]

# SqueezeNet: fire8/squeeze1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire8/expand1x1
[convolutional]
filters=256
size=1
activation=relu

# SqueezeNet: fire8/shortcut
[shortcut]
from=-2
activation=leaky

# SqueezeNet: fire8/expand3x3
[convolutional]
filters=256
size=3
pad=1
activation=relu

# SqueezeNet: fire8/route
[route]
layers=-3,-1

[batchnorm]

# SqueezeNet: fire9/squeeze1x1
[convolutional]
filters=64
size=1
activation=relu

# SqueezeNet: fire9/expand1x1
[convolutional]
filters=256
size=1
activation=relu

# SqueezeNet: fire9/shortcut
[shortcut]
from=-2
activation=leaky

# SqueezeNet: fire9/expand3x3
[convolutional]
filters=256
size=3
pad=1
activation=relu

# SqueezeNet: fire9/route
[route]
layers=-3,-1

# SqueezeNet: drop9
[dropout]
probability=.5

# SqueezeNet: conv10
[convolutional]
filters=1000
size=1
activation=relu

[batchnorm]

# SqueezeNet: pool10
[avgpool]

# YoLo: output = (5 * 2 + CLASSES) * SIDE^2
[connected]
output=539
activation=linear

# YoLo
[detection]
classes=1
coords=4
rescore=1
side=7
num=2
softmax=0
sqrt=1
jitter=.2

object_scale=1
noobject_scale=.5
class_scale=1
coord_scale=5

OR

[net]
batch=64
subdivisions=2
height=227
width=227
channels=3
momentum=0.9
decay=0.0005

learning_rate=0.04
policy=poly
power=1
max_batches=200000

[crop]
crop_width=227
crop_height=227
flip=0
angle=0
saturation=1.5
exposure=1.5

# SqueezeNet: conv1
[convolutional]
batch_normalize=1
filters=64
size=3
stride=2
activation=relu

# SqueezeNet: pool1
[maxpool]
size=3
stride=2

# SqueezeNet: fire2/squeeze1x1
[convolutional]
batch_normalize=1
filters=16
size=1
activation=relu

# SqueezeNet: fire2/expand1x1
[convolutional]
batch_normalize=1
filters=64
size=1
activation=relu

# SqueezeNet: fire2/route
[route]
layers=-2

# SqueezeNet: fire2/expand3x3
[convolutional]
batch_normalize=1
filters=64
size=3
pad=1
activation=relu

# SqueezeNet: fire2/route
[route]
layers=-3,-1

# SqueezeNet: fire3/squeeze1x1
[convolutional]
batch_normalize=1
filters=16
size=1
activation=relu

# SqueezeNet:fire3/expand1x1
[convolutional]
batch_normalize=1
filters=64
size=1
activation=relu

# SqueezeNet: fire3/route
[route]
layers=-2

# SqueezeNet: fire3/expand3x3
[convolutional]
batch_normalize=1
filters=64
size=3
pad=1
activation=relu

# SqueezeNet: fire3/route
[route]
layers=-3,-1

# SqueezeNet: pool3
[maxpool]
size=3
stride=2

# SqueezeNet: fire4/squeeze1x1
[convolutional]
batch_normalize=1
filters=32
size=1
activation=relu

# SqueezeNet: fire4/expand1x1
[convolutional]
batch_normalize=1
filters=128
size=1
activation=relu

# SqueezeNet: fire4/route
[route]
layers=-2

# SqueezeNet: fire4/expand3x3
[convolutional]
batch_normalize=1
filters=128
size=3
pad=1
activation=relu

# SqueezeNet: fire4/route
[route]
layers=-3,-1

# SqueezeNet: fire5/squeeze1x1
[convolutional]
batch_normalize=1
filters=32
size=1
activation=relu

# SqueezeNet: fire5/expand1x1
[convolutional]
batch_normalize=1
filters=128
size=1
activation=relu

# SqueezeNet: fire5/route
[route]
layers=-2

# SqueezeNet: fire5/expand3x3
[convolutional]
batch_normalize=1
filters=128
size=3
pad=1
activation=relu

# SqueezeNet: fire5/route
[route]
layers=-3,-1

# SqueezeNet: pool5
[maxpool]
size=3
stride=2

# SqueezeNet: fire6/squeeze1x1
[convolutional]
batch_normalize=1
filters=48
size=1
activation=relu

# SqueezeNet: fire6/expand1x1
[convolutional]
batch_normalize=1
filters=192
size=1
activation=relu

# SqueezeNet: fire6/route
[route]
layers=-2

# SqueezeNet: fire6/expand3x3
[convolutional]
batch_normalize=1
filters=192
size=3
pad=1
activation=relu

# SqueezeNet: fire6/route
[route]
layers=-3,-1

# SqueezeNet: fire7/squeeze1x1
[convolutional]
batch_normalize=1
filters=48
size=1
activation=relu

# SqueezeNet: fire7/expand1x1
[convolutional]
batch_normalize=1
filters=192
size=1
activation=relu

# SqueezeNet: fire7/route
[route]
layers=-2

# SqueezeNet: fire7/expand3x3
[convolutional]
batch_normalize=1
filters=192
size=3
pad=1
activation=relu

# SqueezeNet: fire7/route
[route]
layers=-3,-1

# SqueezeNet: fire8/squeeze1x1
[convolutional]
batch_normalize=1
filters=64
size=1
activation=relu

# SqueezeNet: fire8/expand1x1
[convolutional]
batch_normalize=1
filters=256
size=1
activation=relu

# SqueezeNet: fire8/route
[route]
layers=-2

# SqueezeNet: fire8/expand3x3
[convolutional]
batch_normalize=1
filters=256
size=3
pad=1
activation=relu

# SqueezeNet: fire8/route
[route]
layers=-3,-1

# SqueezeNet: fire9/squeeze1x1
[convolutional]
batch_normalize=1
filters=64
size=1
activation=relu

# SqueezeNet: fire9/expand1x1
[convolutional]
batch_normalize=1
filters=256
size=1
activation=relu

# SqueezeNet: fire9/route
[route]
layers=-2

# SqueezeNet: fire9/expand3x3
[convolutional]
batch_normalize=1
filters=256
size=3
pad=1
activation=relu

# SqueezeNet: fire9/route
[route]
layers=-3,-1

# SqueezeNet: drop9
[dropout]
probability=.5

# SqueezeNet: conv10
[convolutional]
batch_normalize=1
filters=1000
size=1
activation=relu

# SqueezeNet: pool10
[avgpool]

# YoLo: output = (5 * 2 + CLASSES) * SIDE^2
[connected]
output=1331
activation=linear

# YoLo
[detection]
classes=1
coords=4
rescore=1
side=11
num=2
softmax=0
sqrt=1
jitter=.2

object_scale=1
noobject_scale=.5
class_scale=1
coord_scale=5
pjreddie commented 8 years ago

the avgpool layer will be really bad for detection since you throw out all the spatial information. It's good for classification but you should get rid of it and the last 1000 filter convolutional layer for detection.

Also, just so you know, squeezenet is good in terms of the number of parameters but it's still uses lots of floating point operations. squeezenet uses 2.17 billion operations while the standard darknet config: https://github.com/pjreddie/darknet/blob/master/cfg/darknet.cfg only uses .81 billion operations so it'll probably be faster.

On Wed, Sep 7, 2016 at 9:41 PM, Luiz Vitor Martinez Cardoso < notifications@github.com> wrote:

@pjreddie https://github.com/pjreddie it is good to hear from you buddy!

Please check my latest porting versions of SqueezeNet for Darknet. I had to change route/shortcut layers and use batchnormalization... now loss error is going below 1 on my datasets.

https://bitbucket.org/snippets/geeksys/Ap85M

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DeepScale/SqueezeNet/issues/15#issuecomment-245491956, or mute the thread https://github.com/notifications/unsubscribe-auth/AAScHA08ypMUDnzNC8nd7BMX003d6FCcks5qn5H4gaJpZM4JD4oZ .

Grabber commented 8 years ago

@pjreddie, yes. What about enabling xnor=1? Is it working?

Grabber commented 8 years ago

@pjreddie, by removing 

# SqueezeNet: conv10
[convolutional]
batch_normalize=1
filters=1000
size=1
activation=relu

# SqueezeNet: pool10
[avgpool]

Something went totally wrong:

/data/dataset/006_headtop_outdoor/images/out00017358.jpg
Detection Avg IOU: 0.008419, Pos Cat: 0.222350, All Cat: 0.222350, Pos Obj: -285.093384, Any Obj: 94.234314, count: 41
Detection Avg IOU: 0.005466, Pos Cat: 8.992064, All Cat: 8.992064, Pos Obj: -694.999390, Any Obj: 146.219254, count: 35
5: 1363320704.000000, 245519200.000000 avg, 0.039999 rate, 1.584814 seconds, 320 images

Any idea?

pjreddie commented 8 years ago

You can also check out the tiny-yolo model, it's based on the darknet reference network and it's already trained / i know it works.

http://pjreddie.com/darknet/yolo/#tiny

On Wed, Sep 7, 2016 at 10:35 PM, Luiz Vitor Martinez Cardoso < notifications@github.com> wrote:

@pjreddie https://github.com/pjreddie, by removing

SqueezeNet: conv10

[convolutional] batch_normalize=1 filters=1000 size=1 activation=relu

SqueezeNet: pool10

[avgpool]

Something goes totally wrong:

/data/dataset/006_headtop_outdoor/images/out00017358.jpg Detection Avg IOU: 0.008419, Pos Cat: 0.222350, All Cat: 0.222350, Pos Obj: -285.093384, Any Obj: 94.234314, count: 41 Detection Avg IOU: 0.005466, Pos Cat: 8.992064, All Cat: 8.992064, Pos Obj: -694.999390, Any Obj: 146.219254, count: 35 5: 1363320704.000000, 245519200.000000 avg, 0.039999 rate, 1.584814 seconds, 320 images

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DeepScale/SqueezeNet/issues/15#issuecomment-245498407, or mute the thread https://github.com/notifications/unsubscribe-auth/AAScHADi-sbWj39O03662Wi98vYxnL0xks5qn561gaJpZM4JD4oZ .

Grabber commented 8 years ago

@pjreddie, I got a push notification about your commit and already started training it. Thanks!

ghost commented 8 years ago

@pjreddie What are the upgrades from yolo-tiny to tiny-yolo ? Don't you think removing dropout might impact generalization ? @Grabber Squeezing tiny-yolo might be a good idea, specially with FP16.

ghost commented 8 years ago

@Grabber I think squeezeNet has specially been designed for mimicking AlexNet which natively address the problem of classification. Therefore, as @pjreddie said it, average polling isn't a good idea for detection. I only see an FC layer that could overcome this, but I am afraid the number of parameters would explode and thereby compromise the pruning process.

tmatas commented 7 years ago

Do avgpooling on a cropped area (where the detected object is) might be useful?

Grabber commented 7 years ago

@tmatas that is a question that @pjreddie can answer with much more property, I really don't know.

azuryl commented 6 years ago

@pjreddie Dear pjreddie can you deliver a pretrained Squeezenet model and cfg in darknet so I can comapre the difference between Squeezenet and tiny darknet