Custom initializer for tf weights

[d-a-yakovlev]

Hello there!

I tried to convert model from pytorch to keras and got such thing: An initializer for variable weight of type <dtype: 'complex64'> is required for layer weight_layer. Received: None. So this is common tf problem. And i see https://github.com/AlexanderLutsenko/nobuco/blob/f0f08d91afe26cfe87c5ea90c63d70c48eeea9b6/nobuco/layers/weight.py#L24 https://github.com/AlexanderLutsenko/nobuco/blob/f0f08d91afe26cfe87c5ea90c63d70c48eeea9b6/nobuco/layers/weight.py#L12 here. We don't have any possibility to affect on weight creation.

Could we somehow make possible to push code for custom initializer like this:

def complex_initializer(base_initializer):
    f = base_initializer()

    def initializer(*args, dtype=tf.complex64, **kwargs):
        real = f(*args, **kwargs)
        imag = f(*args, **kwargs)
        return tf.complex(real, imag)

    return initializer

initializer=complex_initializer(tf.random_normal_initializer))

in nobuco.pytorch_to_keras(..., tf_weight_initializer=initializer, ...) call?

self.add_weight('weight', shape=weight_shape, dtype=weight_dtype, initializer=tf_weight_initializer)

Kind regards

[AlexanderLutsenko]

@d-a-yakovlev Hey, thanks for reporting! I just added support for complex numbers in v0.6.0. Let me know if it works for you.

[d-a-yakovlev]

@d-a-yakovlev Hey, thanks for reporting! I just added support for complex numbers in v0.6.0. Let me know if it works for you.

Thanks, but converting is still stuck 😅

Well, i should somehow use trick like this

@nobuco.converter(F.hardsigmoid, channel_ordering_strategy=ChannelOrderingStrategy.MINIMUM_TRANSPOSITIONS)
def hardsigmoid(input: torch.Tensor, inplace: bool = False):
    return lambda input, inplace=False: tf.clip_by_value(input/6 + 1/2, clip_value_min=0, clip_value_max=1)

to solve such problems?

Exception: Unimplemented op: <built-in method atan2 of type object at 0x7f6c5e2a4880>
Exception: Unimplemented op: <built-in method complex of type object at 0x7f6c5e2a4880>
Exception: Unimplemented op: <built-in method group_norm of type object at 0x7f6c5e2a4880>
Exception: Unimplemented op: <built-in method istft of type object at 0x7f6c5e2a4880>
Exception: Unimplemented op: <function glu at 0x7f6ca02a2560>
Exception: Unimplemented op: <method 'all' of 'torch._C._TensorBase' objects>
Exception: Unimplemented op: <method 'std' of 'torch._C._TensorBase' objects>
Exception: Unimplemented op: <method 't' of 'torch._C._TensorBase' objects>
Exception: Unimplemented op: <slot wrapper '__getattribute__' of 'object' objects>

[AlexanderLutsenko]

@d-a-yakovlev Yes, for all these nodes converters must be provided. Most of them are trivial, but __getattribute__ looks suspicious. Would you care to show me your network?

[d-a-yakovlev]

@AlexanderLutsenko Hello, well i stuck with such problems:

/usr/local/lib/python3.10/dist-packages/nobuco/converters/validation.py:66: UserWarning: Validation exception on node 'Sequential': Tensor shapes of output #0 don't match: (Pytorch) [8, 384, 474] vs [8, 768, 237] (Tensorflow)

/usr/local/lib/python3.10/dist-packages/nobuco/converters/validation.py:66: UserWarning: Validation exception on node '__mul__': Tensor shapes of output #0 don't match: (Pytorch) [1, 4, 2, 2048, 474] vs [1, 4, 4, 2048, 474] (Tensorflow) Have you any advice for this?

Well that state i got when i used some plugs on __mul__ and __add__ such:

@nobuco.converter(torch.mul, torch.Tensor.__mul__, torch.Tensor.__imul__, torch.Tensor.__rmul__, channel_ordering_strategy=ChannelOrderingStrategy.MINIMUM_TRANSPOSITIONS_OR_PYTORCH, autocast=True)
def converter_mul(input, other, *, out=None):
    def func(input, other, *, out=None):
        try:
            if (input.shape[0] == other.shape[1] // 2):
                return tf.concat([
                    tf.multiply(other[:,:other.shape[1]//2], input),
                    tf.multiply(other[:,other.shape[1]//2:], input)
                    ], 1)
        except:
            pass
        return input * other
    return func

@nobuco.converter(torch.Tensor.add, torch.Tensor.__add__, torch.Tensor.__iadd__, torch.Tensor.__radd__, channel_ordering_strategy=ChannelOrderingStrategy.MINIMUM_TRANSPOSITIONS_OR_PYTORCH, autocast=True)
def converter_add(input, other, *args, **kwargs):
    def func(input, other, *args, **kwargs):
        try:
            input_n = 1
            for ax in input.shape:
                input_n *= ax
            other_n = 1
            for ax in other.shape:
                other_n *= ax
            if input_n == other_n:
                return input + tf.reshape(other, input.shape)
        except:
            pass
        return input + other
    return func

Otherwise i got:

ValueError: Exception encountered when calling layer "tf.math.multiply_5" (type TFOpLambda).

Dimensions must be equal, but are 48 and 96 for '{{node tf.math.multiply_5/Mul}} = Mul[T=DT_FLOAT](Placeholder, Placeholder_1)' with input shapes: [48,1], [512,96,237].

Call arguments received by layer "tf.math.multiply_5" (type TFOpLambda):
  • x=tf.Tensor(shape=(48, 1), dtype=float32)
  • y=tf.Tensor(shape=(512, 96, 237), dtype=float32)
  • name=None

I'll be grateful for any advice and help 🙏

[AlexanderLutsenko]

@d-a-yakovlev Hi! Try v0.7.0, it implements GroupNorm. Pleases note that Tensorflow requirement has been upped to 2.13. Also, remove the custom __add__ and __mul__ converters, default ones should work just fine.

[d-a-yakovlev]

I have solved all problems. Thanks a lot for help!