approximation modeling problem

[shaieesss]

Dear authors, I want to ask that if I want to approximate the S5 state equation as an encoder layer, like the linear function in PyTorch, how should I do it?

[jimmysmith1919]

Hi thanks for reaching out! Could you please provide a little more detail on what you are looking to do?

[shaieesss]

It is a liitle complicated. Well, I am an academic fresh man, in paper "Imputation-based Time-Series Anomaly Detection with Conditional Weight-Incremental Diffusion Models", I find it uses S4-based U-net as denoising network, inspired by this, I want to find new sequence modeling algorithm as denoising network. Then I find S5. However, in the source code of S4-based U-net, I can't find related ssm module. Only have this:

class FeatureWiseAffine(nn.Module): def init(self, in_channels, out_channels): super(FeatureWiseAffine, self).init() self.noise_func = nn.Sequential( nn.Linear(in_channels, out_channels * (1 + self.use_affine_level))) def forward(self, x, noise_embed): batch = x.shape[0] x = x + self.noise_func(noise_embed).view(batch, -1, 1, 1) return x

This class actually is simple, input added with transformed noise. I think this may be approximation of S4(Other components in U-net module have no relavance to S4 sequence modeling obviously.). So I want to ask if there is approximation of S5.

[jimmysmith1919]

I see. I am not familiar with this paper or their code, but from a quick glimpse, it appears their U-net here https://github.com/ChunjingXiao/DiffAD/blob/main/model/sr3_modules/unet.py, might simply be a U-net based on Attention rather than the S4 U-net they describe in the paper.

I would recommend contacting the authors of this paper and codebase to find out more and to learn if they plan to release their S4 U-net. But I do not believe they are making an approximation.