How can i use codes to implement the 64*64 diffusion model in DDPM-segmentation

[Wanyidon]

In my experiments, i want to use 64*64 image in DDPM-segmentation bue to limited allocation, but code shows "RuntimeError: Error(s) in loading state_dict for UNetModel "

[Wanyidon]

when i use "dim":[256,256,8448] , it occurs [enforce fail at CPUAllocator.cpp:68] . DefaultCPUAllocator: can't allocate memory: you tried to allocate 88583700480 bytes. Error code 12 (Cannot allocate memory)

[dbaranchuk]

This code doesn't support segmentation for 64x64 diffusion models out of the box but it should be easy to adapt it. Just make sure that you create a correct model for 64x64 images and change the configs accordingly. Regarding "dim":[256,256,8448], replace it with [64,64, (your number of features)].

[Wanyidon]

thanks for your reply, i also want to know how could i use my datasets to train a brand new model from the beginning, and my dataset has a bigger size. if i put all my dataset into featrue_extracter it cost immensely cpu memory. So I'm very eager to know the solution.

[dbaranchuk]

preprocess it by batches and store on the disk. Then, one can create a dataset class that access data from the disk without loading all of it at once.

[luccachiang]

This code doesn't support segmentation for 64x64 diffusion models out of the box but it should be easy to adapt it. Just make sure that you create a correct model for 64x64 images and change the configs accordingly. Regarding "dim":[256,256,8448], replace it with [64,64, (your number of features)].

Hi! May I ask how to set the number of features? Or how is your 8448 calculated?

[dbaranchuk]

Hi! To calculate the number of features, you need to sum the feature dims from the corresponding UNet blocks in your model (the block ids are set here) and then multiply it by the number of steps (the steps are here).

[luccachiang]

Sorry if my question seems stupid because I am new to Diffusion models. In the paper it says all the features from the middle blocks are upsampled to the image size and then concate. In this way, why is the feature number here not 256*256? And I assumed that steps only indicate the denoise progress, what is its relationship with features? Thanks in advance for your answer.

[dbaranchuk]

We consider pixel-wise features. The activations after upsampling have 256x256xC dims. Thus, the feature vector for each pixel has a size C.

[CyberAI-XDU]

Hi, I still don't quite understand how 8448 is obtained and what it is used for, can you give me a more detailed explanation? Be grateful