Improvement of `plot_bounding_box_gallery` utility

[innat]

Short Description

Demonstration with visual output is one of the crucial parts. Comparing the following two, I think pycoco-tools exhibit a bit more attractiveness. Maybe mostly for

i. No black border around the box, ii. Box filled with color with lower opacity.

what if we improve from A to B?

with keras_cv (A)	with pycoco_tools (B)

Some other features in the plot_bounding_box_gallery utility should have

• Notwithstanding the different classes, the box color annotation remains the same. It might become important for instance segmentation part.
• Return object (how to set title, x-label, y-label, etc)

Existing Implementations

• https://keras.io/guides/keras_cv/object_detection_keras_cv/

Other Information

The above data set is from: https://www.kaggle.com/datasets/ipythonx/global-wheat-challenge The pycoco-tools is used: https://github.com/cocodataset/cocoapi/tree/master/PythonAPI/pycocotools

cc. @LukeWood

[LukeWood]

Tools like this should aim to be informative in my opinion; not necessarily pretty. Personally I have a hard time finding all boxes on the right due to inconsistency and lack of high contrast lines.

[ianstenbit]

I tend to agree with Luke's comments -- but I do like your point about returning the matplotlib object from our visualization functions. For example, if path is None and not show, we can return plt so that the user can do whatever they want with the plt object manually.

I'll mark this as open for contribution for that change, but I think we should continue to approach visualization with the overarching goal being function, not form.

[innat]

The above wheat example might be a weak example (single class with different color codes).

The motivation came from the following image. The unique object has its own color code. (Also, the box color fill looks super fancy, really not required, but not bad to have an optional argument, i.e. box_fill, etc)

Main objectives:

• Different color-code for unique objects. (make use of class_mapping). (another use cases: instance segmentation.)
• Return the plot object.

---

Combining features from keras-cv and pycoco-tools.

inputs = next(iter(tds.take(1)))
images, bounding_boxes = inputs["images"], inputs["bounding_boxes"]

image_nd = images.numpy().astype(np.uint8)
bbox_nd = bounding_boxes['boxes'].numpy()

get_plt = visualize_bbox(
    image_array=image_nd, 
    boxes_array=bbox_nd, 
    rows=1,
    cols=1,
    scale=10,
    from_type='coco', 
    box_random_color=False,
    box_fill=False
)
get_plt.show()

example1	example2

[Zekrom-7780]

Hello @innat @LukeWood , please could you assign this task to me so that I can try to improve it?

[innat]

@Zekrom-7780

returning the matplotlib object from our visualization functions. For example, if path is None and not show, we can return plt so that the user can do whatever they want with the plt object manually.

You can start working on it. cc. @ianstenbit

[Zekrom-7780]

@ianstenbit @innat Ok, I am SOO sorry for asking such Basic questions, but I need to ask them. I have tried to return the plotted_images after theplot_image_gallery function is called, in the plot_bounding_box_gallery.py, as in

  plot_image_gallery(
          plotted_images,
          value_range,
          legend_handles=legend_handles,
          rows=rows,
          cols=cols,
          **kwargs
      )
  return plotted_images

So How do I like test whatever I have committed in my branch of the forked repository? I did read CONTRIBUTING.md and set up the environment, but couldn't find anything regarding testing this.

Also is there like a slack channel, or mailing lists for asking doubts like this or this place itself is fine?

[ianstenbit]

@Zekrom-7780

returning the matplotlib object from our visualization functions. For example, if path is None and not show, we can return plt so that the user can do whatever they want with the plt object manually.

You can start working on it. cc. @ianstenbit

What exactly is the goal for this issue? To add color-coding to bounding boxes?

[ianstenbit]

@ianstenbit @innat Ok, I am SOO sorry for asking such Basic questions, but I need to ask them. I have tried to return the plotted_images after theplot_image_gallery function is called, in the plot_bounding_box_gallery.py, as in

  plot_image_gallery(
          plotted_images,
          value_range,
          legend_handles=legend_handles,
          rows=rows,
          cols=cols,
          **kwargs
      )
  return plotted_images

So How do I like test whatever I have committed in my branch of the forked repository? I did read CONTRIBUTING.md and set up the environment, but couldn't find anything regarding testing this.

Also is there like a slack channel, or mailing lists for asking doubts like this or this place itself is fine?

I would test this by updating examples/visualization/plot_image_gallery to make use of the new feature and verify that it behaves as-expected.

We do not have a separate communication channel. Asking questions in this issue is fine.

[Zekrom-7780]

@ianstenbit

What exactly is the goal for this issue? To add color-coding to bounding boxes?

Yes, the goal of this issue is to add like different colours to each object that is detected, and to return the matplotlib object for the user, for any reason that they want it to use it.

Am I right?

[Zekrom-7780]

So i am able to return the matplotlib object fig for plot_bounding_box_gallery.py.

Now, for the color code part, @innat , you mentioned that

Different color-code for unique objects. (make use of class_mapping). (another use cases: instance segmentation.)

• So is the idea here that I need to create a new Color Dictionary, with random number of colors (and random values), equal to len(class_mapping), and when each new class is detected, the color of the bounding box should be from the Color dictionary?

• And this work should be in the plot_bounding_box_gallery.py ?

Am i thinking in the right direction? And if not, then how can I use class_mapping?

[innat]

@Zekrom-7780 I raised two feature requests, A. color-code and B. returning plot object. I think taking color code as a feature isn't yet decided or, no perhaps but option B is ok. If you made that adjustment (option B), I think it is okay to send PR. And if option A is okay to take, I can share a starter code.

with keras_cv	with custom plot

[Zekrom-7780]

@innat since Option B is done from my side, i'll send a PR for that And if option A is needed (if it is decided that it is OK) , i'll work on that as well