search

HumanSignal / label-studio

Label Studio is a multi-type data labeling and annotation tool with standardized output format
https://labelstud.io
Apache License 2.0
17.54k stars 2.18k forks source link

Empty Label files when using meta SAM using the label-studio-ml-backend #5607

Open singhsegv opened 4 months ago

singhsegv commented 4 months ago

Describe the bug I've labelled a dataset which has only 1 class for the purpose of semantic segmentation. The annotations are being rendered correctly in the label studio and I am able to export png and numpy binary masks.

But when I try to export the annotations in any of the format:

  1. YOLO: The txt files in the labels are empty
  2. COCO and PASCAL: These formats also aren't giving me any annotations

Expected behavior Annotations should be exportable in all the formats

Environment (please complete the following information):

Additional context

Onandon11 commented 3 months ago

+1 On this, I have same problem. I want do semantic segmentation with the help of SAM. However all segmentation sections are empty after exporting. This problem is a real show stopper, because I have to fall back to manual polygon points, which is unnecessarily tedious.

singhsegv commented 3 months ago

@Onandon11 A workaround that I did was to download the masked images since I only had 1 class and then using opencv to get contours along with some quickly put together python code to manually convert those masks to YOLO and COCO segmentation formats.

Onandon11 commented 3 months ago

But if you export to numpy or png you loose track of the traceability to which photo the mask corresponds - as far as I know the masks get the task ID prepended but not the image name. I thought going down the road to just download the label-studio JSON format and extract the mask there - from the RLE field.

Nevertheless, could you share your code? And did you saw https://github.com/ultralytics/ultralytics/issues/883?

makseq commented 3 months ago

BrushLabels (SAM uses this Label Studio tool) support only numpy or png as export formats at this moment. Or native Label Studio JSON. There were some PRs, e.g.: https://github.com/HumanSignal/label-studio-converter/pull/175/files It enables exporting masks in COCO. But it's still not merged.

Onandon11 commented 3 months ago

Thanks for linking the PR! Hadn't seen it yet.

But maybe a a quick solution for this problem is to change the filenames of the png images to the actual image names of which they were annotated (and included mask name). Then we could script it ourselfs to the proper format. This is now impossible due to what I described in my previous comment.

connor-tan commented 3 months ago

BrushLabels (SAM uses this Label Studio tool) support only numpy or png as export formats at this moment. Or native Label Studio JSON. There were some PRs, e.g.: https://github.com/HumanSignal/label-studio-converter/pull/175/files It enables exporting masks in COCO. But it's still not merged.

how to use this tool? the doc in this project seems so simple

connor-tan commented 3 months ago

Describe the bug I've labelled a dataset which has only 1 class for the purpose of semantic segmentation. The annotations are being rendered correctly in the label studio and I am able to export png and numpy binary masks.

But when I try to export the annotations in any of the format:

  1. YOLO: The txt files in the labels are empty
  2. COCO and PASCAL: These formats also aren't giving me any annotations

Expected behavior Annotations should be exportable in all the formats

Environment (please complete the following information):

  • OS: macOS Ventura 13.6.1 (22G313)
  • Label Studio Version: 1.11.0

Additional context

have you been solved this issue? thx a lot

cmdPromptCritical commented 2 months ago

I, too, am having this issue.

phildoe424 commented 2 months ago

Describe the bug: I'm seeing the same issue where I'm trying to label 3 different classes for image segmentation. After watching demo videos and following the tutorial, I'm excited at the added efficiency in labeling. Previously I've exported to the COCO format using the "Semantic Segmentation with Polygons" template, but I think it makes more sense to use masks with label-studio-ml-backend.

I've tried every export option and haven't been able to export useable masks with any of the export options. YOLO, COCO and PASCAL result in empty segmentation data in the annotation metadata. Both Numpy and PNG options as mentioned previously export masks but there's no programmatic way to determine which mask corresponds to which image.

Environment (please complete the following information): OS: Ubuntu 23.10 (GNU/Linux 6.5.0-28-generic x86_64) Label Studio Version: 1.12.0

ryan-alva-bio commented 3 weeks ago

Still wanting to export SAM labels to COCO at the very least in 2024. Any luck anyone?

PhillipRDI commented 3 weeks ago

Thanks for linking the PR! Hadn't seen it yet.

But maybe a a quick solution for this problem is to change the filenames of the png images to the actual image names of which they were annotated (and included mask name). Then we could script it ourselfs to the proper format. This is now impossible due to what I described in my previous comment.

I implemented a quick solution as you described in a PR (see attached link). Both brush exports in PNG and Numpy will export an images folder and in addition a masks folder where each mask has the same base filename as the original image with an added "-CLASSNAME-N.png". I also am currently working on a conversion script that will take in masks using my filename convention and then generate a Coco compliant results.json including segmentations.

Anyways, if you want to use the mask code that I mentioned the way I've gotten it to work is to follow "Install from source" instructions from label-studio. After everything is installed you'll find the "label_studio_sdk" is located under your home folder at "~/.cache/pypoetry/virtualenvs/label-studio-uZkB5o12-py3.11/lib/python3.11/site-packages/label_studio_sdk". Grab the forked PR code: git clone https://github.com/PhillipRDI/label-studio-sdk.git

The files "converter.py" and "brush.py" from the GIT repo folder label-studio-sdk/src/label_studio_sdk/converter folder should be copied to the ~/.cache/pypoetry/virtualenvs/label-studio-uZkB5o12-py3.11/lib/python3.11/site-packages/label_studio_sdk/converter folder.

PhillipRDI commented 2 weeks ago

MasksToCoco.zip I attached a Jupyter script to convert masks generated from the GIT repo in my last comment to MS Coco format. The first half of this script actually writes out the MS Coco JSON and the last half will just load/display segmentation masks to verify the encodings work.

I noticed that previous request "feat: LSDV-4831: Export BrushLabels to COCO" was closed before completion. It appears there were issues with the Pycocotools dependency. I utilized skimage and pycocotools in the attached Jupyter, but I am a CV engineer and could eliminate this dependency easily using only skimage or opencv. makseq or any other Label Studio admin please advise if you'd like me to create a PR for this feature.