mathias-brandewinder / CNTK.FSharp

MIT License
67 stars 9 forks source link

Completing Conv2D layer #12

Closed mathias-brandewinder closed 6 years ago

mathias-brandewinder commented 6 years ago

CNTK.FSharp/Sequential.fs contains a module Conv2D, with basic blocks to build 2d CNN layers. What arguments or elements are missing to complete that layer? Both the CNTK Python API and Keras can provide hints as to what people would expect to find and that is missing.

AviAvni commented 6 years ago
python f# keras comments
filter_shape Conv2D.Kernel kernel_size optional non-linearity, e.g. activation=relu
num_filters Conv2D.OutputFeatures filters
activation ??? activation
init Conv2D.Initializer kernel_initializer
pad ??? padding if False (default), then the filter will be shifted over the “valid” area of input, that is, no value outside the area is used. If pad is True on the other hand, the filter will be applied to all input positions, and values outside the valid region will be considered zero.
strides ??? strides increment when sliding the filter over the input. E.g. (2,2) to reduce the dimensions by 2
bias ??? use_bias if False, do not include a bias parameter
init_bias ??? bias_initializer initializer for the bias
use_correlation ??? ??? currently always True and cannot be changed. It indicates that Convolution() actually computes the cross-correlation rather than the true convolution
??? ??? data_format A string, one of channels_last (default) or channels_first. The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch, height, width, channels) while channels_first corresponds to inputs with shape (batch, channels, height, width). It defaults to the image_data_format value found in your Keras config file at ~/.keras/keras.json. If you never set it, then it will be "channels_last".
??? ??? dilation_rate an integer or tuple/list of 2 integers, specifying the dilation rate to use for dilated convolution. Can be a single integer to specify the same value for all spatial dimensions. Currently, specifying any dilation_rate value != 1 is incompatible with specifying any stride value != 1.
??? ??? kernel_regularizer Regularizer function applied to the kernel weights matrix (see regularizer).
??? ??? bias_regularizer Regularizer function applied to the bias vector (see regularizer).
??? ??? activity_regularizer Regularizer function applied to the output of the layer (its "activation"). (see regularizer).
??? ??? kernel_constraint Constraint function applied to the kernel matrix (see constraints).
??? ??? bias_constraint Constraint function applied to the bias vector (see constraints).
AviAvni commented 6 years ago
python f# keras comments
MaxPooling Pool2D.PoolingType = PoolingType.Max MaxPooling2D
AveragePooling PoolingType.Average AveragePooling2D
filter_shape Pool2D.Window pool_size
strides Pool2D.Stride strides
pad ??? padding
??? ??? data_format
python f# keras comments
GlobalMaxPooling ??? GlobalMaxPooling2D
GlobalAveragePooling ??? GlobalAveragePooling2D
AviAvni commented 6 years ago

@mathias-brandewinder this line https://github.com/mathias-brandewinder/CNTK.FSharp/blob/master/CNTK.FSharp/Sequential.fs#L128 is the strides parameter for conv layer and now it's not configurable do you want me to add this parameter to Conv2D?

mathias-brandewinder commented 6 years ago

That sounds like the right way to go! Thank you :)

mathias-brandewinder commented 6 years ago

Done, not 100% parity, but close enough: commit #17