Fmap size 16 works, whereas Fmap size of 8 doesn't work

[snoop2head]

https://github.com/lucidrains/DALLE-pytorch/blob/4a7958dc0c3313dee91dda5603741139ee5483e3/dalle_pytorch/vae.py#L178

[lucidrains]

why wouldn't it work? what's the full error you are seeing?

[snoop2head]

Overview

• Will replicate the error and attach error log as soon as possible.
• Compatibility error occured when I used VQGAN trained with fmap size(or attn_resolutions in VQGAN config) of 8.

Personal Speculation

It seems like num_layers and fmap size defined in two different classes(VQGanVAE and DALLE) caused the problem.

For the fmap size and num_layers in vae.py, it is stated as the following:

https://github.com/lucidrains/DALLE-pytorch/blob/4a7958dc0c3313dee91dda5603741139ee5483e3/dalle_pytorch/vae.py#L177-L178

• I loaded VQGAN model where attention resolution size is set to 8.
• Image resolution is 256.
• Thus value of f is 32 and num_layers is 5.

Whereas in dalle_pytorch.py, image_fmap_size is the following:

https://github.com/lucidrains/DALLE-pytorch/blob/1ad3ab89898a58c4de83c1e82c830f302bbee07c/dalle_pytorch/dalle_pytorch.py#L337

• vae.image_size is set to 256 as default.
• image_fmap_size becomes 256 // (2 ** 5) = 8

Code-wise they should coincide with the one another, but error log showed that it doesn't.

Will attach the log and code here for the reference.