DFCAN 3D

[akas0018]

What changes would one need to make to the DFCAN 2D notebook for it to be able to handle 3D datasets. I see that the key components are already written for handling 3D data. Has anyone given this a try ?

[esgomezm]

Maybe @iarganda or @IvanHCenalmor know a bit more about it?

[IvanHCenalmor]

Hi @akas0018,

I have been looking the at DFCAN 2D notebook and in order to make the notebook to be able to handle 3D datasets some changes in the code are needed. Those changes can be divided in two.

First of all, even if the key components are written, the final model and some components need some changes: https://github.com/HenriquesLab/ZeroCostDL4Mic/blob/77feded97957441b1d998432c349c77b2dcc17b2/Colab_notebooks/Beta%20notebooks/DFCAN_ZeroCostDL4Mic.ipynb?short_path=2eef267#L497 https://github.com/HenriquesLab/ZeroCostDL4Mic/blob/77feded97957441b1d998432c349c77b2dcc17b2/Colab_notebooks/Beta%20notebooks/DFCAN_ZeroCostDL4Mic.ipynb?short_path=2eef267#L508 https://github.com/HenriquesLab/ZeroCostDL4Mic/blob/77feded97957441b1d998432c349c77b2dcc17b2/Colab_notebooks/Beta%20notebooks/DFCAN_ZeroCostDL4Mic.ipynb?short_path=2eef267#L518 https://github.com/HenriquesLab/ZeroCostDL4Mic/blob/77feded97957441b1d998432c349c77b2dcc17b2/Colab_notebooks/Beta%20notebooks/DFCAN_ZeroCostDL4Mic.ipynb?short_path=2eef267#L526

These components need to be adapted to the changes explained in the Note 6 from the Supplementary information of the original paper. Therefore a nearly new model has to be implemented.

Secondly, the function that create_random_patches which is defined in https://github.com/HenriquesLab/ZeroCostDL4Mic/blob/77feded97957441b1d998432c349c77b2dcc17b2/Colab_notebooks/Beta%20notebooks/DFCAN_ZeroCostDL4Mic.ipynb?short_path=2eef267#L310 and then used in https://github.com/HenriquesLab/ZeroCostDL4Mic/blob/77feded97957441b1d998432c349c77b2dcc17b2/Colab_notebooks/Beta%20notebooks/DFCAN_ZeroCostDL4Mic.ipynb?short_path=2eef267#L1730 has to be adapted. The following lines of code are only adapted to create 2D patches:

  input_patches.append(lr_img[ r : r + lr_shape[0],  
                               c : c + lr_shape[1] ] ) 
  output_patches.append(hr_img[ r*scale : (r + lr_shape[0])*scale, 
                                c*scale : (c + lr_shape[1])*scale ] )`

I don't really know how the patches work in 3D data, if the third dimension is also reduced for the patch or if it is completely taken but those lines need to be changed.

Finally, the code cell from section 4.1 has two lines of code related with the input and output shapes, that should be changed: https://github.com/HenriquesLab/ZeroCostDL4Mic/blob/77feded97957441b1d998432c349c77b2dcc17b2/Colab_notebooks/Beta%20notebooks/DFCAN_ZeroCostDL4Mic.ipynb?short_path=2eef267#L1750 https://github.com/HenriquesLab/ZeroCostDL4Mic/blob/77feded97957441b1d998432c349c77b2dcc17b2/Colab_notebooks/Beta%20notebooks/DFCAN_ZeroCostDL4Mic.ipynb?short_path=2eef267#L1756 At this moment, they are also fixed to two dimensions with a single channel, this should be adapted to your data or make it modular with something like, for example:

input_shape = train_patches_wf[0].shape[1:] + (1,)
output_shape = train_patches_gt[0].shape[1:] + (1,)

A desired input shape (the one that you should try to obtain) based on tf.keras.layers.Conv3D should be 5+D tensor with shape: batch_shape + (channels, conv_dim1, conv_dim2, conv_dim3) if data_format='channels_first' or 5+D tensor with shape: batch_shape + (conv_dim1, conv_dim2, conv_dim3, channels) if data_format='channels_last'.

This would only be until section 4 included, for sections 5 and 6 maybe other changes should be made for the reading of the 3D data, but the changes would be similar to the ones described.

I hope this helps you if you desire to implement a 3D version of the DFCAN notebook :)

[esgomezm]

@akas0018 did this help?