PPYOLOFPN结构修改

[zsbjmy]

我在PaddleDetection/ppdet/modeling/necks/yolo_fpn.py 做了如下修改： 【在forward中加入 block = ChannelAttention(block)； block = SpatialAttention()】

class PPYOLOFPN(nn.Layer):
    __shared__ = ['norm_type', 'data_format']

    def __init__(self,
                 in_channels=[512, 1024, 2048],
                 norm_type='bn',
                 data_format='NCHW',
                 coord_conv=False,
                 conv_block_num=2,
                 drop_block=False,
                 block_size=3,
                 keep_prob=0.9,
                 spp=False):
        """
        PPYOLOFPN layer

        Args:
            in_channels (list): input channels for fpn
            norm_type (str): batch norm type, default bn
            data_format (str): data format, NCHW or NHWC
            coord_conv (bool): whether use CoordConv or not
            conv_block_num (int): conv block num of each pan block
            drop_block (bool): whether use DropBlock or not
            block_size (int): block size of DropBlock
            keep_prob (float): keep probability of DropBlock
            spp (bool): whether use spp or not

        """
        super(PPYOLOFPN, self).__init__()
        assert len(in_channels) > 0, "in_channels length should > 0"
        self.in_channels = in_channels
        self.num_blocks = len(in_channels)
        # parse kwargs
        self.coord_conv = coord_conv
        self.drop_block = drop_block
        self.block_size = block_size
        self.keep_prob = keep_prob
        self.spp = spp
        self.conv_block_num = conv_block_num
        self.data_format = data_format
        if self.coord_conv:
            ConvLayer = CoordConv
        else:
            ConvLayer = ConvBNLayer

        if self.drop_block:
            dropblock_cfg = [[
                'dropblock', DropBlock, [self.block_size, self.keep_prob],
                dict()
            ]]
        else:
            dropblock_cfg = []

        self._out_channels = []
        self.yolo_blocks = []
        self.routes = []
        for i, ch_in in enumerate(self.in_channels[::-1]):
            if i > 0:
                ch_in += 512 // (2**i)
            channel = 64 * (2**self.num_blocks) // (2**i)
            base_cfg = []
            c_in, c_out = ch_in, channel
            for j in range(self.conv_block_num):
                base_cfg += [
                    [
                        'conv{}'.format(2 * j), ConvLayer, [c_in, c_out, 1],
                        dict(
                            padding=0, norm_type=norm_type)
                    ],
                    [
                        'conv{}'.format(2 * j + 1), ConvBNLayer,
                        [c_out, c_out * 2, 3], dict(
                            padding=1, norm_type=norm_type)
                    ],
                ]
                c_in, c_out = c_out * 2, c_out

            base_cfg += [[
                'route', ConvLayer, [c_in, c_out, 1], dict(
                    padding=0, norm_type=norm_type)
            ], [
                'tip', ConvLayer, [c_out, c_out * 2, 3], dict(
                    padding=1, norm_type=norm_type)
            ]]

            if self.conv_block_num == 2:
                if i == 0:
                    if self.spp:
                        spp_cfg = [[
                            'spp', SPP, [channel * 4, channel, 1], dict(
                                pool_size=[5, 9, 13], norm_type=norm_type)
                        ]]
                    else:
                        spp_cfg = []
                    cfg = base_cfg[0:3] + spp_cfg + base_cfg[
                        3:4] + dropblock_cfg + base_cfg[4:6]
                else:
                    cfg = base_cfg[0:2] + dropblock_cfg + base_cfg[2:6]
            elif self.conv_block_num == 0:
                if self.spp and i == 0:
                    spp_cfg = [[
                        'spp', SPP, [c_in * 4, c_in, 1], dict(
                            pool_size=[5, 9, 13], norm_type=norm_type)
                    ]]
                else:
                    spp_cfg = []
                cfg = spp_cfg + dropblock_cfg + base_cfg
            name = 'yolo_block.{}'.format(i)
            yolo_block = self.add_sublayer(name, PPYOLODetBlock(cfg, name))
            self.yolo_blocks.append(yolo_block)
            self._out_channels.append(channel * 2)
            if i < self.num_blocks - 1:
                name = 'yolo_transition.{}'.format(i)
                route = self.add_sublayer(
                    name,
                    ConvBNLayer(
                        ch_in=channel,
                        ch_out=256 // (2**i),
                        filter_size=1,
                        stride=1,
                        padding=0,
                        norm_type=norm_type,
                        data_format=data_format,
                        name=name))
                self.routes.append(route)

    def forward(self, blocks):
        assert len(blocks) == self.num_blocks
        blocks = blocks[::-1]
        yolo_feats = []
        for i, block in enumerate(blocks):
            if i > 0:
                if self.data_format == 'NCHW':
                    logger.info("进入ChannelA")
                    block = ChannelAttention(block)
                    block = SpatialAttention()
                    block = paddle.concat([route, block], axis=1)
                else:
                    block = ChannelAttention(block)
                    block = SpatialAttention()
                    block = paddle.concat([route, block], axis=-1)
            route, tip = self.yolo_blocks[i](block)
            yolo_feats.append(tip)

            if i < self.num_blocks - 1:
                route = self.routes[i](route)
                route = F.interpolate(
                    route, scale_factor=2., data_format=self.data_format)

        return yolo_feats

    @classmethod
    def from_config(cls, cfg, input_shape):
        return {'in_channels': [i.channels for i in input_shape], }

    @property
    def out_shape(self):
        return [ShapeSpec(channels=c) for c in self._out_channels]

class ChannelAttention(nn.Layer):
    def __init__(self, in_planes, ratio=16):
        super(ChannelAttention, self).__init__()
        self.avg_pool = nn.AdaptiveAvgPool2D(1)
        self.max_pool = nn.AdaptiveAvgPool2D(1)

        self.fc1   = nn.Conv2D(in_planes, in_planes // 16, 1, bias=False)
        self.relu1 = F.relu()
        self.fc2   = nn.Conv2D(in_planes // 16, in_planes, 1, bias=False)

        self.sigmoid = F.sigmoid()

    def forward(self, x):
        logger.info("进入ChannelAttention")

        avg_out = self.fc2(self.relu1(self.fc1(self.avg_pool(x))))
        max_out = self.fc2(self.relu1(self.fc1(self.max_pool(x))))
        out = avg_out + max_out
        return self.sigmoid(out)

class SpatialAttention(nn.Layer):
    def __init__(self, kernel_size=7):
        super(SpatialAttention, self).__init__()
        logger.info("进入SpatialAttention")

        assert kernel_size in (3, 7), 'kernel size must be 3 or 7'
        padding = 3 if kernel_size == 7 else 1

        self.conv1 = nn.Conv2D(2, 1, kernel_size, padding=padding, bias=False)
        self.sigmoid = F.sigmoid()

    def forward(self, x):
        avg_out = paddle.mean(x, dim=1, keepdim=True)
        max_out, _ = paddle.max(x, dim=1, keepdim=True)
        x = paddle.concat([avg_out, max_out], dim=1)
        x = self.conv1(x)
        return self.sigmoid(x)

[yghstill]

@zsbjmy good job. 欢迎提交PR

[zsbjmy]

您好，请问我这样修改是有效的吗？我的logger.info("进入ChannelA")在notebook中没有打印出来，是不是说明没有进入到我添加的 ChannelAttention(nn.Layer)函数中？

[wangxinxin08]

这种写法是无效的，所有的模块都需要在init函数中初始化，另外SpitialAttention的结构好像也有点问题，这个模块的输出channel直接变成1了，我理解你现在实现的SpitialAttention的输出只是注意力分数

[zsbjmy]

我在PaddleDetection/ppdet/modeling/necks/yolo_fpn.py 这个文件中直接加入了ChannelAttention(nn.Layer)这个函数，

import paddle import paddle.nn as nn import paddle.nn.functional as F from paddle import ParamAttr from ppdet.core.workspace import register, serializable from ..backbones.darknet import ConvBNLayer import numpy as np

from ..shape_spec import ShapeSpec

from .logger import setup_logger logger = setup_logger(name)

all = ['YOLOv3FPN', 'PPYOLOFPN']

def add_coord(x, data_format): b = x.shape[0] if data_format == 'NCHW': h = x.shape[2] w = x.shape[3] else: h = x.shape[1] w = x.shape[2]

gx = paddle.arange(w, dtype='float32') / (w - 1.) * 2.0 - 1.
if data_format == 'NCHW':
    gx = gx.reshape([1, 1, 1, w]).expand([b, 1, h, w])
else:
    gx = gx.reshape([1, 1, w, 1]).expand([b, h, w, 1])
gx.stop_gradient = True

gy = paddle.arange(h, dtype='float32') / (h - 1.) * 2.0 - 1.
if data_format == 'NCHW':
    gy = gy.reshape([1, 1, h, 1]).expand([b, 1, h, w])
else:
    gy = gy.reshape([1, h, 1, 1]).expand([b, h, w, 1])
gy.stop_gradient = True

return gx, gy

class YoloDetBlock(nn.Layer): def init(self, ch_in, channel, norm_type, name, data_format='NCHW'): """ YOLODetBlock layer for yolov3, see https://arxiv.org/abs/1804.02767

    Args:
        ch_in (int): input channel
        channel (int): base channel
        norm_type (str): batch norm type
        name (str): layer name
        data_format (str): data format, NCHW or NHWC
    """
    super(YoloDetBlock, self).__init__()
    self.ch_in = ch_in
    self.channel = channel
    assert channel % 2 == 0, \
        "channel {} cannot be divided by 2".format(channel)
    conv_def = [
        ['conv0', ch_in, channel, 1, '.0.0'],
        ['conv1', channel, channel * 2, 3, '.0.1'],
        ['conv2', channel * 2, channel, 1, '.1.0'],
        ['conv3', channel, channel * 2, 3, '.1.1'],
        ['route', channel * 2, channel, 1, '.2'],
    ]

    self.conv_module = nn.Sequential()
    for idx, (conv_name, ch_in, ch_out, filter_size,
              post_name) in enumerate(conv_def):
        self.conv_module.add_sublayer(
            conv_name,
            ConvBNLayer(
                ch_in=ch_in,
                ch_out=ch_out,
                filter_size=filter_size,
                padding=(filter_size - 1) // 2,
                norm_type=norm_type,
                data_format=data_format,
                name=name + post_name))

    self.tip = ConvBNLayer(
        ch_in=channel,
        ch_out=channel * 2,
        filter_size=3,
        padding=1,
        norm_type=norm_type,
        data_format=data_format,
        name=name + '.tip')

def forward(self, inputs):
    route = self.conv_module(inputs)
    tip = self.tip(route)
    return route, tip

class SPP(nn.Layer): def init(self, ch_in, ch_out, k, pool_size, norm_type, name, act='leaky', data_format='NCHW'): """ SPP layer, which consist of four pooling layer follwed by conv layer

    Args:
        ch_in (int): input channel of conv layer
        ch_out (int): output channel of conv layer
        k (int): kernel size of conv layer
        norm_type (str): batch norm type
        name (str): layer name
        data_format (str): data format, NCHW or NHWC
    """
    super(SPP, self).__init__()
    self.pool = []
    self.data_format = data_format
    for size in pool_size:
        pool = self.add_sublayer(
            '{}.pool1'.format(name),
            nn.MaxPool2D(
                kernel_size=size,
                stride=1,
                padding=size // 2,
                data_format=data_format,
                ceil_mode=False))
        self.pool.append(pool)
    self.conv = ConvBNLayer(
        ch_in,
        ch_out,
        k,
        padding=k // 2,
        norm_type=norm_type,
        name=name,
        act=act,
        data_format=data_format)

def forward(self, x):
    outs = [x]
    for pool in self.pool:
        outs.append(pool(x))
    if self.data_format == "NCHW":
        y = paddle.concat(outs, axis=1)
    else:
        y = paddle.concat(outs, axis=-1)

    y = self.conv(y)
    return y

class DropBlock(nn.Layer): def init(self, block_size, keep_prob, name, data_format='NCHW'): """ DropBlock layer, see https://arxiv.org/abs/1810.12890

    Args:
        block_size (int): block size
        keep_prob (int): keep probability
        name (str): layer name
        data_format (str): data format, NCHW or NHWC
    """
    super(DropBlock, self).__init__()
    self.block_size = block_size
    self.keep_prob = keep_prob
    self.name = name
    self.data_format = data_format

def forward(self, x):
    if not self.training or self.keep_prob == 1:
        return x
    else:
        gamma = (1. - self.keep_prob) / (self.block_size**2)
        if self.data_format == 'NCHW':
            shape = x.shape[2:]
        else:
            shape = x.shape[1:3]
        for s in shape:
            gamma *= s / (s - self.block_size + 1)

        matrix = paddle.cast(paddle.rand(x.shape, x.dtype) < gamma, x.dtype)
        mask_inv = F.max_pool2d(
            matrix,
            self.block_size,
            stride=1,
            padding=self.block_size // 2,
            data_format=self.data_format)
        mask = 1. - mask_inv
        y = x * mask * (mask.numel() / mask.sum())
        return y

class CoordConv(nn.Layer): def init(self, ch_in, ch_out, filter_size, padding, norm_type, name, data_format='NCHW'): """ CoordConv layer

    Args:
        ch_in (int): input channel
        ch_out (int): output channel
        filter_size (int): filter size, default 3
        padding (int): padding size, default 0
        norm_type (str): batch norm type, default bn
        name (str): layer name
        data_format (str): data format, NCHW or NHWC

    """
    super(CoordConv, self).__init__()
    self.conv = ConvBNLayer(
        ch_in + 2,
        ch_out,
        filter_size=filter_size,
        padding=padding,
        norm_type=norm_type,
        data_format=data_format,
        name=name)
    self.data_format = data_format

def forward(self, x):
    gx, gy = add_coord(x, self.data_format)
    if self.data_format == 'NCHW':
        y = paddle.concat([x, gx, gy], axis=1)
    else:
        y = paddle.concat([x, gx, gy], axis=-1)
    y = self.conv(y)
    return y

class PPYOLODetBlock(nn.Layer): def init(self, cfg, name, data_format='NCHW'): """ PPYOLODetBlock layer

    Args:
        cfg (list): layer configs for this block
        name (str): block name
        data_format (str): data format, NCHW or NHWC
    """
    super(PPYOLODetBlock, self).__init__()
    self.conv_module = nn.Sequential()
    for idx, (conv_name, layer, args, kwargs) in enumerate(cfg[:-1]):
        kwargs.update(
            name='{}.{}'.format(name, conv_name), data_format=data_format)
        self.conv_module.add_sublayer(conv_name, layer(*args, **kwargs))

    conv_name, layer, args, kwargs = cfg[-1]
    kwargs.update(
        name='{}.{}'.format(name, conv_name), data_format=data_format)
    self.tip = layer(*args, **kwargs)

def forward(self, inputs):
    route = self.conv_module(inputs)
    tip = self.tip(route)
    return route, tip

class PPYOLOTinyDetBlock(nn.Layer): def init(self, ch_in, ch_out, name, drop_block=False, block_size=3, keep_prob=0.9, data_format='NCHW'): """ PPYOLO Tiny DetBlock layer Args: ch_in (list): input channel number ch_out (list): output channel number name (str): block name drop_block: whether user DropBlock block_size: drop block size keep_prob: probability to keep block in DropBlock data_format (str): data format, NCHW or NHWC """ super(PPYOLOTinyDetBlock, self).init() self.dropblock = drop_block self.conv_module = nn.Sequential()

    cfgs = [
        # name, in channels, out channels, filter_size, 
        # stride, padding, groups
        ['.0', ch_in, ch_out, 1, 1, 0, 1],
        ['.1', ch_out, ch_out, 5, 1, 2, ch_out],
        ['.2', ch_out, ch_out, 1, 1, 0, 1],
        ['.route', ch_out, ch_out, 5, 1, 2, ch_out],
    ]
    for cfg in cfgs:
        conv_name, conv_ch_in, conv_ch_out, filter_size, stride, padding, \
                groups = cfg
        self.conv_module.add_sublayer(
            name + conv_name,
            ConvBNLayer(
                ch_in=conv_ch_in,
                ch_out=conv_ch_out,
                filter_size=filter_size,
                stride=stride,
                padding=padding,
                groups=groups,
                name=name + conv_name))

    self.tip = ConvBNLayer(
        ch_in=ch_out,
        ch_out=ch_out,
        filter_size=1,
        stride=1,
        padding=0,
        groups=1,
        name=name + conv_name)

    if self.drop_block_:
        self.drop_block = DropBlock(
            block_size=block_size,
            keep_prob=keep_prob,
            data_format=data_format,
            name=name + '.dropblock')

def forward(self, inputs):
    if self.drop_block_:
        inputs = self.drop_block(inputs)
    route = self.conv_module(inputs)
    tip = self.tip(route)
    return route, tip

class PPYOLODetBlockCSP(nn.Layer): def init(self, cfg, ch_in, ch_out, act, norm_type, name, data_format='NCHW'): """ PPYOLODetBlockCSP layer

    Args:
        cfg (list): layer configs for this block
        ch_in (int): input channel
        ch_out (int): output channel
        act (str): default mish
        name (str): block name
        data_format (str): data format, NCHW or NHWC
    """
    super(PPYOLODetBlockCSP, self).__init__()
    self.data_format = data_format
    self.conv1 = ConvBNLayer(
        ch_in,
        ch_out,
        1,
        padding=0,
        act=act,
        norm_type=norm_type,
        name=name + '.left',
        data_format=data_format)
    self.conv2 = ConvBNLayer(
        ch_in,
        ch_out,
        1,
        padding=0,
        act=act,
        norm_type=norm_type,
        name=name + '.right',
        data_format=data_format)
    self.conv3 = ConvBNLayer(
        ch_out * 2,
        ch_out * 2,
        1,
        padding=0,
        act=act,
        norm_type=norm_type,
        name=name,
        data_format=data_format)
    self.conv_module = nn.Sequential()
    for idx, (layer_name, layer, args, kwargs) in enumerate(cfg):
        kwargs.update(name=name + layer_name, data_format=data_format)
        self.conv_module.add_sublayer(layer_name, layer(*args, **kwargs))

def forward(self, inputs):
    conv_left = self.conv1(inputs)
    conv_right = self.conv2(inputs)
    conv_left = self.conv_module(conv_left)
    if self.data_format == 'NCHW':
        conv = paddle.concat([conv_left, conv_right], axis=1)
    else:
        conv = paddle.concat([conv_left, conv_right], axis=-1)

    conv = self.conv3(conv)
    return conv, conv

@register @serializable class YOLOv3FPN(nn.Layer): shared = ['norm_type', 'data_format']

def __init__(self,
             in_channels=[512, 1024, 2048],
             norm_type='bn',
             data_format='NCHW'):
    """
    YOLOv3FPN layer

    Args:
        in_channels (list): input channels for fpn
        norm_type (str): batch norm type, default bn
        data_format (str): data format, NCHW or NHWC

    """
    super(YOLOv3FPN, self).__init__()
    assert len(in_channels) > 0, "in_channels length should > 0"
    self.in_channels = in_channels
    self.num_blocks = len(in_channels)

    self._out_channels = []
    self.yolo_blocks = []
    self.routes = []
    self.data_format = data_format
    for i in range(self.num_blocks):
        name = 'yolo_block.{}'.format(i)
        in_channel = in_channels[-i - 1]
        if i > 0:
            in_channel += 512 // (2**i)
        yolo_block = self.add_sublayer(
            name,
            YoloDetBlock(
                in_channel,
                channel=512 // (2**i),
                norm_type=norm_type,
                data_format=data_format,
                name=name))
        self.yolo_blocks.append(yolo_block)
        # tip layer output channel doubled
        self._out_channels.append(1024 // (2**i))
        # if(i==2) {

        # }else{
        #     self._out_channels.append(1024 // (2**i))
        # }

        if i < self.num_blocks - 1:
            name = 'yolo_transition.{}'.format(i)
            route = self.add_sublayer(
                name,
                ConvBNLayer(
                    ch_in=512 // (2**i),
                    ch_out=256 // (2**i),
                    filter_size=1,
                    stride=1,
                    padding=0,
                    norm_type=norm_type,
                    data_format=data_format,
                    name=name))
            self.routes.append(route)

def forward(self, blocks):
    assert len(blocks) == self.num_blocks
    blocks = blocks[::-1]
    print(blocks)
    yolo_feats = []
    for i, block in enumerate(blocks):
        if i > 0:
            if self.data_format == 'NCHW':
                block = paddle.concat([route, block], axis=1)
            else:
                block = paddle.concat([route, block], axis=-1)
        route, tip = self.yolo_blocks[i](block)
        yolo_feats.append(tip)
        # if(i==2){

        # }
        # else{
        #     yolo_feats.append(tip)
        #     yolo_feats.append(tip)
        #     yolo_feats.append(tip)
        # }

        if i < self.num_blocks - 1:
            route = self.routes[i](route)
            route = F.interpolate(
                route, scale_factor=2., data_format=self.data_format) 

    print(yolo_feats)
    return yolo_feats

@classmethod
def from_config(cls, cfg, input_shape):
    return {'in_channels': [i.channels for i in input_shape], }

@property
def out_shape(self):
    return [ShapeSpec(channels=c) for c in self._out_channels]

@register @serializable class PPYOLOFPN(nn.Layer): shared = ['norm_type', 'data_format']

def __init__(self,
             in_channels=[512, 1024, 2048],
             norm_type='bn',
             data_format='NCHW',
             coord_conv=False,
             conv_block_num=2,
             drop_block=False,
             block_size=3,
             keep_prob=0.9,
             spp=False):
    """
    PPYOLOFPN layer

    Args:
        in_channels (list): input channels for fpn
        norm_type (str): batch norm type, default bn
        data_format (str): data format, NCHW or NHWC
        coord_conv (bool): whether use CoordConv or not
        conv_block_num (int): conv block num of each pan block
        drop_block (bool): whether use DropBlock or not
        block_size (int): block size of DropBlock
        keep_prob (float): keep probability of DropBlock
        spp (bool): whether use spp or not

    """
    super(PPYOLOFPN, self).__init__()
    assert len(in_channels) > 0, "in_channels length should > 0"
    self.in_channels = in_channels
    self.num_blocks = len(in_channels)
    # parse kwargs
    self.coord_conv = coord_conv
    self.drop_block = drop_block
    self.block_size = block_size
    self.keep_prob = keep_prob
    self.spp = spp
    self.conv_block_num = conv_block_num
    self.data_format = data_format
    if self.coord_conv:
        ConvLayer = CoordConv
    else:
        ConvLayer = ConvBNLayer

    if self.drop_block:
        dropblock_cfg = [[
            'dropblock', DropBlock, [self.block_size, self.keep_prob],
            dict()
        ]]
    else:
        dropblock_cfg = []

    self._out_channels = []
    self.yolo_blocks = []
    self.routes = []
    for i, ch_in in enumerate(self.in_channels[::-1]):
        if i > 0:
            ch_in += 512 // (2**i)
        channel = 64 * (2**self.num_blocks) // (2**i)
        base_cfg = []
        c_in, c_out = ch_in, channel
        for j in range(self.conv_block_num):
            base_cfg += [
                [
                    'conv{}'.format(2 * j), ConvLayer, [c_in, c_out, 1],
                    dict(
                        padding=0, norm_type=norm_type)
                ],
                [
                    'conv{}'.format(2 * j + 1), ConvBNLayer,
                    [c_out, c_out * 2, 3], dict(
                        padding=1, norm_type=norm_type)
                ],
            ]
            c_in, c_out = c_out * 2, c_out

        base_cfg += [[
            'route', ConvLayer, [c_in, c_out, 1], dict(
                padding=0, norm_type=norm_type)
        ], [
            'tip', ConvLayer, [c_out, c_out * 2, 3], dict(
                padding=1, norm_type=norm_type)
        ]]

        if self.conv_block_num == 2:
            if i == 0:
                if self.spp:
                    spp_cfg = [[
                        'spp', SPP, [channel * 4, channel, 1], dict(
                            pool_size=[5, 9, 13], norm_type=norm_type)
                    ]]
                else:
                    spp_cfg = []
                cfg = base_cfg[0:3] + spp_cfg + base_cfg[
                    3:4] + dropblock_cfg + base_cfg[4:6]
            else:
                cfg = base_cfg[0:2] + dropblock_cfg + base_cfg[2:6]
        elif self.conv_block_num == 0:
            if self.spp and i == 0:
                spp_cfg = [[
                    'spp', SPP, [c_in * 4, c_in, 1], dict(
                        pool_size=[5, 9, 13], norm_type=norm_type)
                ]]
            else:
                spp_cfg = []
            cfg = spp_cfg + dropblock_cfg + base_cfg
        name = 'yolo_block.{}'.format(i)
        yolo_block = self.add_sublayer(name, PPYOLODetBlock(cfg, name))
        self.yolo_blocks.append(yolo_block)
        self._out_channels.append(channel * 2)
        if i < self.num_blocks - 1:
            name = 'yolo_transition.{}'.format(i)
            route = self.add_sublayer(
                name,
                ConvBNLayer(
                    ch_in=channel,
                    ch_out=256 // (2**i),
                    filter_size=1,
                    stride=1,
                    padding=0,
                    norm_type=norm_type,
                    data_format=data_format,
                    name=name))
            self.routes.append(route)

def forward(self, blocks):
    assert len(blocks) == self.num_blocks
    blocks = blocks[::-1]
    yolo_feats = []
    for i, block in enumerate(blocks):
        if i > 0:
            if self.data_format == 'NCHW':
                logger.info("进入ChannelA")
                block = ChannelAttention(block)
                block = SpatialAttention()
                block = paddle.concat([route, block], axis=1)
            else:
                block = ChannelAttention(block)
                block = SpatialAttention()
                block = paddle.concat([route, block], axis=-1)
        route, tip = self.yolo_blocks[i](block)
        yolo_feats.append(tip)

        if i < self.num_blocks - 1:
            route = self.routes[i](route)
            route = F.interpolate(
                route, scale_factor=2., data_format=self.data_format)

    return yolo_feats

@classmethod
def from_config(cls, cfg, input_shape):
    return {'in_channels': [i.channels for i in input_shape], }

@property
def out_shape(self):
    return [ShapeSpec(channels=c) for c in self._out_channels]

@register @serializable class PPYOLOTinyFPN(nn.Layer): shared = ['norm_type', 'data_format']

def __init__(self,
             in_channels=[80, 56, 34],
             detection_block_channels=[160, 128, 96],
             norm_type='bn',
             data_format='NCHW',
             **kwargs):
    """
    PPYOLO Tiny FPN layer
    Args:
        in_channels (list): input channels for fpn
        detection_block_channels (list): channels in fpn
        norm_type (str): batch norm type, default bn
        data_format (str): data format, NCHW or NHWC
        kwargs: extra key-value pairs, such as parameter of DropBlock and spp 
    """
    super(PPYOLOTinyFPN, self).__init__()
    assert len(in_channels) > 0, "in_channels length should > 0"
    self.in_channels = in_channels[::-1]
    assert len(detection_block_channels
               ) > 0, "detection_block_channelslength should > 0"
    self.detection_block_channels = detection_block_channels
    self.data_format = data_format
    self.num_blocks = len(in_channels)
    # parse kwargs
    self.drop_block = kwargs.get('drop_block', False)
    self.block_size = kwargs.get('block_size', 3)
    self.keep_prob = kwargs.get('keep_prob', 0.9)

    self.spp_ = kwargs.get('spp', False)
    if self.spp_:
        self.spp = SPP(self.in_channels[0] * 4,
                       self.in_channels[0],
                       k=1,
                       pool_size=[5, 9, 13],
                       norm_type=norm_type,
                       name='spp')

    self._out_channels = []
    self.yolo_blocks = []
    self.routes = []
    for i, (
            ch_in, ch_out
    ) in enumerate(zip(self.in_channels, self.detection_block_channels)):
        name = 'yolo_block.{}'.format(i)
        if i > 0:
            ch_in += self.detection_block_channels[i - 1]
        yolo_block = self.add_sublayer(
            name,
            PPYOLOTinyDetBlock(
                ch_in,
                ch_out,
                name,
                drop_block=self.drop_block,
                block_size=self.block_size,
                keep_prob=self.keep_prob))
        self.yolo_blocks.append(yolo_block)
        self._out_channels.append(ch_out)

        if i < self.num_blocks - 1:
            name = 'yolo_transition.{}'.format(i)
            route = self.add_sublayer(
                name,
                ConvBNLayer(
                    ch_in=ch_out,
                    ch_out=ch_out,
                    filter_size=1,
                    stride=1,
                    padding=0,
                    norm_type=norm_type,
                    data_format=data_format,
                    name=name))
            self.routes.append(route)

def forward(self, blocks):
    assert len(blocks) == self.num_blocks
    blocks = blocks[::-1]

    yolo_feats = []
    for i, block in enumerate(blocks):
        if i == 0 and self.spp_:
            block = self.spp(block)

        if i > 0:
            if self.data_format == 'NCHW':
                block = paddle.concat([route, block], axis=1)
            else:
                block = paddle.concat([route, block], axis=-1)
        route, tip = self.yolo_blocks[i](block)
        yolo_feats.append(tip)

        if i < self.num_blocks - 1:
            route = self.routes[i](route)
            route = F.interpolate(
                route, scale_factor=2., data_format=self.data_format)

    return yolo_feats

@classmethod
def from_config(cls, cfg, input_shape):
    return {'in_channels': [i.channels for i in input_shape], }

@property
def out_shape(self):
    return [ShapeSpec(channels=c) for c in self._out_channels]

@register @serializable class PPYOLOPAN(nn.Layer): shared = ['norm_type', 'data_format']

def __init__(self,
             in_channels=[512, 1024, 2048],
             norm_type='bn',
             data_format='NCHW',
             act='mish',
             conv_block_num=3,
             drop_block=False,
             block_size=3,
             keep_prob=0.9,
             spp=False):
    """
    PPYOLOPAN layer with SPP, DropBlock and CSP connection.

    Args:
        in_channels (list): input channels for fpn
        norm_type (str): batch norm type, default bn
        data_format (str): data format, NCHW or NHWC
        act (str): activation function, default mish
        conv_block_num (int): conv block num of each pan block
        drop_block (bool): whether use DropBlock or not
        block_size (int): block size of DropBlock
        keep_prob (float): keep probability of DropBlock
        spp (bool): whether use spp or not

    """
    super(PPYOLOPAN, self).__init__()
    assert len(in_channels) > 0, "in_channels length should > 0"
    self.in_channels = in_channels
    self.num_blocks = len(in_channels)
    # parse kwargs
    self.drop_block = drop_block
    self.block_size = block_size
    self.keep_prob = keep_prob
    self.spp = spp
    self.conv_block_num = conv_block_num
    self.data_format = data_format
    if self.drop_block:
        dropblock_cfg = [[
            'dropblock', DropBlock, [self.block_size, self.keep_prob],
            dict()
        ]]
    else:
        dropblock_cfg = []

    # fpn
    self.fpn_blocks = []
    self.fpn_routes = []
    fpn_channels = []
    for i, ch_in in enumerate(self.in_channels[::-1]):
        if i > 0:
            ch_in += 512 // (2**(i - 1))
        channel = 512 // (2**i)
        base_cfg = []
        for j in range(self.conv_block_num):
            base_cfg += [
                # name, layer, args
                [
                    '{}.0'.format(j), ConvBNLayer, [channel, channel, 1],
                    dict(
                        padding=0, act=act, norm_type=norm_type)
                ],
                [
                    '{}.1'.format(j), ConvBNLayer, [channel, channel, 3],
                    dict(
                        padding=1, act=act, norm_type=norm_type)
                ]
            ]

        if i == 0 and self.spp:
            base_cfg[3] = [
                'spp', SPP, [channel * 4, channel, 1], dict(
                    pool_size=[5, 9, 13], act=act, norm_type=norm_type)
            ]

        cfg = base_cfg[:4] + dropblock_cfg + base_cfg[4:]
        name = 'fpn.{}'.format(i)
        fpn_block = self.add_sublayer(
            name,
            PPYOLODetBlockCSP(cfg, ch_in, channel, act, norm_type, name,
                              data_format))
        self.fpn_blocks.append(fpn_block)
        fpn_channels.append(channel * 2)
        if i < self.num_blocks - 1:
            name = 'fpn_transition.{}'.format(i)
            route = self.add_sublayer(
                name,
                ConvBNLayer(
                    ch_in=channel * 2,
                    ch_out=channel,
                    filter_size=1,
                    stride=1,
                    padding=0,
                    act=act,
                    norm_type=norm_type,
                    data_format=data_format,
                    name=name))
            self.fpn_routes.append(route)
    # pan
    self.pan_blocks = []
    self.pan_routes = []
    self._out_channels = [512 // (2**(self.num_blocks - 2)), ]
    for i in reversed(range(self.num_blocks - 1)):
        name = 'pan_transition.{}'.format(i)
        route = self.add_sublayer(
            name,
            ConvBNLayer(
                ch_in=fpn_channels[i + 1],
                ch_out=fpn_channels[i + 1],
                filter_size=3,
                stride=2,
                padding=1,
                act=act,
                norm_type=norm_type,
                data_format=data_format,
                name=name))
        self.pan_routes = [route, ] + self.pan_routes
        base_cfg = []
        ch_in = fpn_channels[i] + fpn_channels[i + 1]
        channel = 512 // (2**i)
        for j in range(self.conv_block_num):
            base_cfg += [
                # name, layer, args
                [
                    '{}.0'.format(j), ConvBNLayer, [channel, channel, 1],
                    dict(
                        padding=0, act=act, norm_type=norm_type)
                ],
                [
                    '{}.1'.format(j), ConvBNLayer, [channel, channel, 3],
                    dict(
                        padding=1, act=act, norm_type=norm_type)
                ]
            ]

        cfg = base_cfg[:4] + dropblock_cfg + base_cfg[4:]
        name = 'pan.{}'.format(i)
        pan_block = self.add_sublayer(
            name,
            PPYOLODetBlockCSP(cfg, ch_in, channel, act, norm_type, name,
                              data_format))

        self.pan_blocks = [pan_block, ] + self.pan_blocks
        self._out_channels.append(channel * 2)

    self._out_channels = self._out_channels[::-1]

def forward(self, blocks):
    assert len(blocks) == self.num_blocks
    blocks = blocks[::-1]
    # fpn
    fpn_feats = []
    for i, block in enumerate(blocks):
        if i > 0:
            if self.data_format == 'NCHW':
                block = paddle.concat([route, block], axis=1)
            else:
                block = paddle.concat([route, block], axis=-1)
        route, tip = self.fpn_blocks[i](block)
        fpn_feats.append(tip)

        if i < self.num_blocks - 1:
            route = self.fpn_routes[i](route)
            route = F.interpolate(
                route, scale_factor=2., data_format=self.data_format)

    pan_feats = [fpn_feats[-1], ]
    route = fpn_feats[self.num_blocks - 1]
    for i in reversed(range(self.num_blocks - 1)):
        block = fpn_feats[i]
        route = self.pan_routes[i](route)
        if self.data_format == 'NCHW':
            block = paddle.concat([route, block], axis=1)
        else:
            block = paddle.concat([route, block], axis=-1)

        route, tip = self.pan_blocks[i](block)
        pan_feats.append(tip)

    return pan_feats[::-1]

@classmethod
def from_config(cls, cfg, input_shape):
    return {'in_channels': [i.channels for i in input_shape], }

@property
def out_shape(self):
    return [ShapeSpec(channels=c) for c in self._out_channels]

class ChannelAttention(nn.Layer): def init(self, in_planes, ratio=16): super(ChannelAttention, self).init() self.avg_pool = nn.AdaptiveAvgPool2D(1) self.max_pool = nn.AdaptiveAvgPool2D(1)

    self.fc1   = nn.Conv2D(in_planes, in_planes // 16, 1, bias=False)
    self.relu1 = F.relu()
    self.fc2   = nn.Conv2D(in_planes // 16, in_planes, 1, bias=False)

    self.sigmoid = F.sigmoid()

def forward(self, x):
    logger.info("进入ChannelAttention")

    avg_out = self.fc2(self.relu1(self.fc1(self.avg_pool(x))))
    max_out = self.fc2(self.relu1(self.fc1(self.max_pool(x))))
    out = avg_out + max_out
    return self.sigmoid(out)

class SELayer(nn.Layer): def init(self, ch, reductionratio=16): super(SELayer, self).init() self.pool = nn.AdaptiveAvgPool2D(1) stdv = 1.0 / math.sqrt(ch) c = ch // reductionratio self.squeeze = nn.Linear( ch, c, weight_attr=paddle.ParamAttr(initializer=Uniform(-stdv, stdv)), bias_attr=True)

    stdv = 1.0 / math.sqrt(c_)
    self.extract = nn.Linear(
        c_,
        ch,
        weight_attr=paddle.ParamAttr(initializer=Uniform(-stdv, stdv)),
        bias_attr=True)

def forward(self, inputs):
    out = self.pool(inputs)
    out = paddle.squeeze(out, axis=[2, 3])
    out = self.squeeze(out)
    out = F.relu(out)
    out = self.extract(out)
    out = F.sigmoid(out)
    out = paddle.unsqueeze(out, axis=[2, 3])
    scale = out * inputs
    return scale

class SpatialAttention(nn.Layer): def init(self, kernel_size=7): super(SpatialAttention, self).init() logger.info("进入SpatialAttention")

    assert kernel_size in (3, 7), 'kernel size must be 3 or 7'
    padding = 3 if kernel_size == 7 else 1

    self.conv1 = nn.Conv2D(2, 1, kernel_size, padding=padding, bias=False)
    self.sigmoid = F.sigmoid()

def forward(self, x):
    avg_out = paddle.mean(x, dim=1, keepdim=True)
    max_out, _ = paddle.max(x, dim=1, keepdim=True)
    x = paddle.concat([avg_out, max_out], dim=1)
    x = self.conv1(x)
    return self.sigmoid(x)

[zsbjmy]

还需要在初始化中倒入模块吗？

[wangxinxin08]

需要在初始化中倒入模块，所有继承自nn.Layer类都需要在初始化中倒入模块，类似nn.Conv2D等内置层

[zsbjmy]

[zsbjmy]

您帮我看看，这样初始化对吗？ 在AT.py中有我想添加的类SELayer(),我在init()中 from . import AT , from .AT import * 在yolo_fpn.py中 from .AT import ChannelAttention, SELayer, SpatialAttention 最后在ppyoloFPN的forward()中使用了SELayer（） 但是，notebook中依然没有打印出logger.info('Hello ZS SELayer')内容，请问我该怎么办？

[wangxinxin08]

不对，上面已经说过了在init函数中初始化

[zsbjmy]

[wangxinxin08]

这个写法是ok的

[zsbjmy]

非常感谢！请问在init()函数中的下面这句，参数block是任意变量，还是必须在上面代码中出现过的变量？ if self.se: self.at = SELayer(block)

[zsbjmy]

我这个self.at = SELayer(block)中的block实际上是骨干网络输出的特征层，但是在init()函数中没有对应的变量

[zsbjmy]

在forward()中有骨干网络输出的特征层 def forward(self, blocks): assert len(blocks) == self.num_blocks blocks = blocks[::-1] yolo_feats = [] for i, block in enumerate(blocks): if i > 0: if self.data_format == 'NCHW': if self.se: logger.info('Hello ZS SELayer') block = self.at(block) block = paddle.concat([route, block], axis=1) else: block = paddle.concat([route, block], axis=1) else: if self.se: logger.info('Hello ZS SELayer') block = self.at(block) block = paddle.concat([route, block], axis=-1) else: block = paddle.concat([route, block], axis=-1)

        route, tip = self.yolo_blocks[i](block)
        yolo_feats.append(tip)

[zsbjmy]

请问为什么我的print语句的内容在notebook中打印不出来，我想看代码的中间参数的内容，但是，无论是用logger.info（）还是用print（）语句都不能进行打印？

[wangxinxin08]

在其他位置能打印吗？

[zsbjmy]

其他位置也不能打印print

[zsbjmy]

我怀疑在PaddleDetection/ppdet/modeling/necks/yolo_fpn.py这个文件里改代码不起作用，因为我将PPYOLOFPN这个类整体注释掉，模型依然能够跑起来

[wangxinxin08]

那你用到PPYOLOFPN这个类了吗？你运行的是什么模型？

[zsbjmy]

architecture: YOLOv3 pretrain_weights: https://paddledet.bj.bcebos.com/models/pretrained/ResNet50_vd_ssld_pretrained.pdparams norm_type: sync_bn use_ema: true ema_decay: 0.9998

YOLOv3: backbone: ResNet neck: PPYOLOFPN yolo_head: YOLOv3Head post_process: BBoxPostProcess 用到了

[zsbjmy]

请问我想改源代码，应该怎么办？

[wangxinxin08]

1. 先确保你改的代码和你运行的代码是同一个路径下的代码
2. 确保你代码改完之后保存了
3. 最后可以清除下pycache文件夹

[zsbjmy]

非常感谢您的帮助！

1. 我想和您确认一下，修改PaddleDetection/ppdet/modeling/necks/yolo_fpn.py这个文件里的代码确实可以对源代码进行修改，对吗？
2. 请问pycache这个文件夹的具体路径能帮我指明吗？

[wangxinxin08]

1. 可以对源码进行修改
2. find . -name "pycache" 可以搜索下，一般当前目录下

[paddle-bot-old[bot]]

Since this issue has not been updated for more than three months, it will be closed, if it is not solved or there is a follow-up one, please reopen it at any time and we will continue to follow up. It is recommended to pull and try the latest code first. 由于该问题超过三个月未更新，将会被关闭，若问题未解决或有后续问题，请随时重新打开（建议先拉取最新代码进行尝试），我们会继续跟进。