Open suvadityamuk opened 1 year ago
Would we want to allow custom backbones? If so, how would we do that?
That's planned for object detection, and IMO, should be for semantic segmentation too if not already. In #1128, the default backbone is a ResNet101, but the user can supply a keras_cv.models.Model
and define the layer names to be used for low-level and high-level feature maps.
For example:
backbone = keras_cv.models...
model = segmentation.DeepLabV3Plus(
classes=11,
include_rescaling=True,
backbone=backbone,
feature_layers=("v2_stack_1_block4_1_relu", "v2_stack_3_block2_2_relu"),
input_shape=(256, 256, 3),
)
Would we want to allow custom backbones? If so, how would we do that?
That's planned for object detection, and IMO, should be for semantic segmentation too if not already. In #1128, the default backbone is a ResNet101, but the user can supply a
keras_cv.models.Model
and define the layer names to be used for low-level and high-level feature maps.For example:
backbone = keras_cv.models... model = segmentation.DeepLabV3Plus( classes=11, include_rescaling=True, backbone=backbone, feature_layers=("v2_stack_1_block4_1_relu", "v2_stack_3_block2_2_relu"), input_shape=(256, 256, 3), )
Alright, makes sense. Are we sure we want to go ahead with this?
Would we want to allow custom backbones? If so, how would we do that?
That's planned for object detection, and IMO, should be for semantic segmentation too if not already. In #1128, the default backbone is a ResNet101, but the user can supply a
keras_cv.models.Model
and define the layer names to be used for low-level and high-level feature maps. For example:backbone = keras_cv.models... model = segmentation.DeepLabV3Plus( classes=11, include_rescaling=True, backbone=backbone, feature_layers=("v2_stack_1_block4_1_relu", "v2_stack_3_block2_2_relu"), input_shape=(256, 256, 3), )
Alright, makes sense. Are we sure we want to go ahead with this?
We would like all our meta architectures (detection architectures, segmentation architectures, and any future architectures) to take custom backbones
We would like all our meta architectures (detection architectures, segmentation architectures, and any future architectures) to take custom backbones
Is there any specification on how we should accept the backbone? Or should we use @DavidLandup0 's method?
We would like all our meta architectures (detection architectures, segmentation architectures, and any future architectures) to take custom backbones
Is there any specification on how we should accept the backbone? Or should we use @DavidLandup0 's method?
You can follow the examples under keras_cv/models/object_detection
and keras_cv/models/segmentation
Alright, makes sense. Thank you!
So our current implementation is looking at adding the VGG16 and VGG19 models as backbones. Should we look at any other model backbones?
So our current implementation is looking at adding the VGG16 and VGG19 models as backbones. Should we look at any other model backbones?
given this is modular design, you can start with whatever backbone that is available. We have been using keras.applications backbones but is migrating to keras_cv.models backbones
Makes sense. The VGG16 backbone is now available as in keras_cv.models
as part of #1164
For now, I'll use both keras.applications backbones, but we can look for a refactor in the future.
Sending a PR your way with a preliminary API implementation in a while. Training script and tests still remain though. Will work on it once the API is confirmed.
Short Description A fresh issue to discuss the implementation of the Fully Convolutional Network for Semantic Segmentation paper Last discussed here in #1099 As mentioned in Roadmap for Q4 2022
Papers Fully Convolutional Network for Semantic Segmentation - 34479 citations, 222 mentions (from arXiv scite_ tool)
Results from paper for task : Semantic Segmentation
Existing Implementations
Other Information I think we should have a discussion on the API for the model. Backbones can vary significantly for this model although an almost-standardized method uses the VGG16 and VGG19 models.
@IMvision12 has already begun working on a draft implementation for the VGG16 backbone. We will be collaborating and co-authoring this model.
We can keep the ResNet-50, VGG16 and VGG19 models as supported backbones for the model with their model configs present for use.
Would we want to allow custom backbones? If so, how would we do that?
tf.keras.Model
instance passed as a parameter, which would presumably be a plain CNN.depth_kernel_sizes
,depth_num_filters
, anddepth_padding_modes
which would be then used to generate the backend in an iterative fashion.tf.keras.layers.Conv2D
/tf.keras.Sequential
which would themselves be the backbones' implementation@bhack @ianstenbit @LukeWood @tanzhenyu