tianyu-z
commented
4 years ago Some explanation:
-
How to I get 2 outputs from the deeplab encoder: It's not wise to change the encoder architecture because all the tasks will be affected. I decided to make changes to the maskGen decoder. Instead of only getting the output_x from the deeplab encoder, I also need intermediate layers as the inputs of the decoder. So I did these changes to the encoder
Thus every a piece of code that looks like self.z = self.G.encode(x) was changed into self.z = self.G.encode(x)[2]
The maskDecoder is changed into this one (which was just BaseDecoder before):
-
The result of the new maskDecoder was not ranging from 0 to 1. Thus, the torch.sigmoid() before bceloss is needed.
-
The result of the new maskDecoder has the size: [batch_size, 2, H, W] but the main_loss and TVLoss in "m" task are designed to serve [batch_size, 1, H, W]. Thus, I did this: prediction_main[:, 0, :, :].unsqueeze(1) which change the [batch_size, 2, H, W] to [batch_size, 1, H, W].
-
The output of the new maskDecoder size is [batch_size, 2, 80, 80] which is not [batch_size, 2, new_size, new_size], where new_size is 640 here. Thus, I added upsamplers: nn.Upsample(size=(self.output_size, self.output_size), mode="bilinear", align_corners=True)
This is now a runable but really slow version. I will try to make it faster while waiting for the experiment result this week.
Multilevel advent (640*640): 7.1s/step which is pretty slow.
In contrast singlelevel advent (640*640): 4.2s/step