Custom Pooling Layer for CNN

[Rivilyo-Sitanggang]

I want to make a custom 2x2 Pooling Layer for my CNN architecture with this function with Keras

def cantor_pairing(a, b): return (a + b) * (a + b + 1) // 2 + b

i want the function to get this input : [a, b c, d]

and return cantor_pairing(cantor_pairing(a, b), cantor_pairing(c, d))

But somehow it always give me error. Can anyone please give me a hand on this problems?

[mehtamansi29]

Hi @Rivilyo-Sitanggang -

Here using model subclassing you can create custom 2x2 pooling layer along with your function. Here is sample code for that:

def cantor_pairing(a, b): 
  return (a + b) * (a + b + 1) // 2 + b
class Custom_pooling_layer(keras.layers.Layer):
  def __init__(self,**kwargs):
    super(Custom_pooling_layer, self).__init__(**kwargs)

  def call(self,inputs):
    a, b,c,d = inputs[:, 0], inputs[:, 1],inputs[:, 2], inputs[:, 3]

    result = cantor_pairing(cantor_pairing(a, b), cantor_pairing(c, d))
    return result

Here can find model details about model subclassing.

Also here is the sample custom_pooling function as well:

def custom_pooling(input_shape,pool_size,stride): 
  output= np.floor((input_shape - pool_size) / stride) + 1
  return output.astype('int') 

[Rivilyo-Sitanggang]

When I put the custom layer to my model

# Cantor_CNN model
model = Sequential()

model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))
model.add(Custom_pooling_layer())
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(Custom_pooling_layer())
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(10, activation='softmax'))

# Compile model
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

it gives this error

ValueError: Input 0 of layer "conv2d_4" is incompatible with the layer: expected min_ndim=4, found ndim=3. Full shape received: (None, 26, 32)

[mehtamansi29]

Hi @Rivilyo-Sitanggang -

The error you are facing because of your Custom_pooling_layer class expecting minimum 4 dimension input. By expand_dims of output of Custom_pooling_layer class will resolve this error.

def cantor_pairing(a, b):
  return (a + b) * (a + b + 1) // 2 + b

class Custom_pooling_layer(keras.layers.Layer):
  def __init__(self,**kwargs):
    super(Custom_pooling_layer, self).__init__(**kwargs)
  def call(self,inputs):
    a, b,c,d = inputs[:, 0], inputs[:, 1],inputs[:, 2], inputs[:, 3]
    result = cantor_pairing(cantor_pairing(a, b), cantor_pairing(c, d))
    result = keras.ops.expand_dims(result, axis=-1)#Added expand_dimesion for getting 4 dimesion output
    return result

Attached gist for the reference.

[github-actions[bot]]

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

[Rivilyo-Sitanggang]

I did some modification in my custom pooling layer. this modification did not give any error, but the loss and accuracy somehow does not change at all. i'm sure that this happened because the gradient does not works correctly, but i don't know how to fix this issue.

For the function it take 2 integers input and map it to 1 unique integer (this also explains why there is a lot of normalization process). I repeat the process several times so the function will map 4 integers to 1 unique integer. I already put the custom gradient based on the equation, but I don't know why the training process doesn't feel like it's "training"

def MinMaxNormalization(inputs, min_value, max_value, new_max):
    normalized = (inputs - min_value) / (max_value - min_value) * new_max
    return tf.stop_gradient(normalized)

# Custom Cantor pairing function with custom gradient
def cantor_pairing(a, b):
    result = (a + b) * (a + b + 1) // 2 + b
    return tf.stop_gradient(result)

@tf.custom_gradient
def cantor_pooling(a, b, c ,d):
    a = MinMaxNormalization(a, 0, 1, 255) 
    b = MinMaxNormalization(b, 0, 1, 255)
    c = MinMaxNormalization(c, 0, 1, 255)
    d = MinMaxNormalization(d, 0, 1, 255)

    cantor1 = tf.round(cantor_pairing(a, b))
    cantor1 = MinMaxNormalization(cantor1, 0, 130560, 255)

    cantor2 = tf.round(cantor_pairing(c, d))
    cantor2 = MinMaxNormalization(cantor2, 0, 130560, 255)

    result = cantor_pairing(cantor1, cantor2)
    result = MinMaxNormalization(result, 0, 130560, 1)

    def grad(dy):
        da = dy*(a + b + 0.5)
        db = dy*(a + b + 1.5)
        dc = dy*(c + d + 0.5)
        dd = dy*(c + d + 1.5)
        return da, db, dc, dd

    return result, grad

# Pooling layer with Cantor pairing using TensorFlow operations
class CustomPoolingLayer(tf.keras.layers.Layer):
    def __init__(self, **kwargs):
        super(CustomPoolingLayer, self).__init__(**kwargs)

    def call(self, inputs):
        batch_size = tf.shape(inputs)[0]
        height = tf.shape(inputs)[1]
        width = tf.shape(inputs)[2]

        new_height = height // 2
        new_width = width // 2

        pooled_output = tf.TensorArray(tf.float32, size=new_height * new_width)

        for i in range(new_height):
            for j in range(new_width):
                a = inputs[:, 2 * i, 2 * j, 0]
                b = inputs[:, 2 * i, 2 * j + 1, 0]
                c = inputs[:, 2 * i + 1, 2 * j, 0]
                d = inputs[:, 2 * i + 1, 2 * j + 1, 0]

                pooled_val = cantor_pooling(a,b,c,d)
                pooled_output = pooled_output.write(i * new_width + j, pooled_val)

        pooled_output = pooled_output.stack()
        pooled_output = tf.reshape(pooled_output, (batch_size, new_height, new_width, 1))

        return pooled_output