Errors In Changing Annotation Types

[StupiddCupid]

Hey agml team,

I'd like to generate bounding boxes instead of segmentations, so I changed the opt annotation_ type to be "agml.synthetic.AnnotationType.object_detection". But here I got an error when generating the images:

Thanks!

[ctyeong]

I have the same issue.

[masonearles]

@amogh7joshi Any ideas on this?

On Wed, Feb 8, 2023 at 11:03 Taeyeong Choi @.***> wrote:

I have the same issue.

— Reply to this email directly, view it on GitHub https://github.com/Project-AgML/AgML/issues/44#issuecomment-1423100357, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG5FGPH6NJ53IC6ZUCPYTJTWWPUXPANCNFSM6AAAAAAUUSMDOU . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[amogh7joshi]

I've seen this issue before; it might have something to do with the annotations. Can you re-run the generation with

generator.generate(name = 'Plant4', num_images = 2, convert_dataset = False)

and then print out the contents of the annotation text files in the generated directory? It might be there are text files with empty annotations, or perhaps something else.

[ctyeong]

@amogh7joshi Yes, it works with: generator.generate(..., convert_data = False).

Bounding boxes seem to be written in txt files though. Would you plan to fix this out for right conversion shortly? Or not soon?

[ctyeong]

@amogh7joshi Do you think this is a right output of bounding box annotation? I guess something wrong here ... I tried using some of your codes for conversion to the COCO format: https://github.com/Project-AgML/AgML/blob/d4fc61a76c8f940a78ebb3d2b9a8b1decfc25075/agml/synthetic/converter.py

Output when using "rectangular_labels_branches.txt":

My code:

with open(path, 'r') as f:
    annotations = np.array([line.replace('\n', '').strip().split(' ') for line in f.readlines()])
    print(annotations.shape)
    annotations = annotations[:, 1:].astype(np.float32)

# Convert the bounding boxes to COCO JSON format.
x_c, y_c, w, h = np.rollaxis(annotations, 1)
x_min = (x_c - w / 2) * img.size[0]
y_min = ((1 - y_c) - h / 2) * img.size[1]

w = w * img.size[0]
h = h * img.size[1]
bbs = np.dstack([x_min, y_min, w, h])[0].astype(np.int32)

# visualize 
fig, ax = plt.subplots()
ax.imshow(np.array(img)) 

for bb in bbs: 
    ax.add_patch(Rectangle((bb[1], bb[0]), bb[3], bb[2],
             edgecolor = 'red', fill=False, lw=2))

Output with "rectangular_labels_clusters.txt"

Please let me know if I should have taken extra care to re-use your code.

Updated: It was my fault with the indices of bb for visualization. Those should have been:

ax.add_patch(Rectangle((bb[0], bb[1]), bb[2], bb[3], 
             edgecolor = 'red', fill=False, lw=2))

[ctyeong]

@StupiddCupid @amogh7joshi @masonearles I've realized how to fix https://github.com/Project-AgML/AgML/issues/44#issue-1575189667 and make AgML finally work for bounding box annotation. A couple of modifications on converter.py seem to be needed as below.

1. Check whether there is no annotations at all

https://github.com/Project-AgML/AgML/blob/d4fc61a76c8f940a78ebb3d2b9a8b1decfc25075/agml/synthetic/converter.py#L223 An error occurred if no bounding box was present in an image. So, I've added if like below.

  # Read the text file and get all of the lines in float format.
  with open(path, 'r') as f:
      annotations = np.array(
          [line.replace('\n', '').strip().split(' ') for line in f.readlines()])

  if len(annotations) > 0: # <- Added 
    annotations = annotations[:, 1:].astype(np.float32)

    # Convert the bounding boxes to COCO JSON format.
    x_c, y_c, w, h = np.rollaxis(annotations, 1)
    x_min = (x_c - w / 2) * width
    y_min = ((1 - y_c) - h / 2) * height
    w = w * width
    h = h * height
    coords = np.dstack([x_min, y_min, w, h])[0].astype(np.int32)

    # Update the bounding box dictionary.
    bboxes[indx + 1] = coords

  # Return the bounding box dictionary.
  return bboxes

2. Swap width and height.

Somehow, they were wrong in order when assigned at https://github.com/Project-AgML/AgML/blob/d4fc61a76c8f940a78ebb3d2b9a8b1decfc25075/agml/synthetic/converter.py#L203 So, I had to swap them to get correct locations of bounding boxes: width, height = self._meta.image_size

Example output:

+ One more swap

This is not about bounding boxes, but the generated annotations.json also contained width and height of image in a wrong order. So I had to change https://github.com/Project-AgML/AgML/blob/d4fc61a76c8f940a78ebb3d2b9a8b1decfc25075/agml/synthetic/converter.py#L170 to

file_name': fpath, 'width': self._meta.image_size[0],
                'height': self._meta.image_size[1], 'id': indx + 1})

Hopefully, those are considered in the next update.

[masonearles]

Hi Taeyong. Please feel to fix the code and make a pull request in a new branch. That will help speed up the release of the bug fix. Thanks!

On Sat, Feb 11, 2023 at 00:49 Taeyeong Choi @.***> wrote:

@amogh7joshi https://github.com/amogh7joshi @masonearles https://github.com/masonearles I've realized how to make AgML finally work for bounding box annotation by modifying a couple of things on converter.py as below.

1. Check whether there is no annotations at all

https://github.com/Project-AgML/AgML/blob/d4fc61a76c8f940a78ebb3d2b9a8b1decfc25075/agml/synthetic/converter.py#L223 An error occurred if no bounding box was present in an image. So, I've added if like below.

Read the text file and get all of the lines in float format.

with open(path, 'r') as f: annotations = np.array( [line.replace('\n', '').strip().split(' ') for line in f.readlines()])

if len(annotations) > 0: # <- Added annotations = annotations[:, 1:].astype(np.float32)

# Convert the bounding boxes to COCO JSON format.
x_c, y_c, w, h = np.rollaxis(annotations, 1)
x_min = (x_c - w / 2) * width
y_min = ((1 - y_c) - h / 2) * height
w = w * width
h = h * height
coords = np.dstack([x_min, y_min, w, h])[0].astype(np.int32)

# Update the bounding box dictionary.
bboxes[indx + 1] = coords

Return the bounding box dictionary.

return bboxes

2. Swap width and height.

Somehow, they were wrong in order when assigned at https://github.com/Project-AgML/AgML/blob/d4fc61a76c8f940a78ebb3d2b9a8b1decfc25075/agml/synthetic/converter.py#L203 So, I had to swap them to get correct locations of bounding boxes: width, height = self._meta.image_size

Example output: [image: image] https://user-images.githubusercontent.com/7784087/218248938-19421627-3829-4f13-9094-87db67df71be.png

3. One more swap

This is not about bounding boxes, but the generated annotations.json also contained width and height of image in a wrong order. So I had to change

https://github.com/Project-AgML/AgML/blob/d4fc61a76c8f940a78ebb3d2b9a8b1decfc25075/agml/synthetic/converter.py#L170 to

file_name': fpath, 'width': self._meta.image_size[0], 'height': self._meta.image_size[1], 'id': indx + 1})

Hopefully, those are considered in the next update.

— Reply to this email directly, view it on GitHub https://github.com/Project-AgML/AgML/issues/44#issuecomment-1426664486, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG5FGPDXEGB3ZTMGNXRZN5TWW5HCHANCNFSM6AAAAAAUUSMDOU . You are receiving this because you were mentioned.Message ID: @.***>

[ctyeong]

@masonearles Pull request done! Thanks!

[amogh7joshi]

Thanks for your help, Taeyong. I've merged in the updates. I'm closing this branch now as I think the issue is resolved, but feel free to re-open it (or a new one) if something else related to this comes up.