Closed LinasKo closed 3 months ago
I found there's some tasks I overlooked.
In addition to examples on their Hugging Face page, they have 5-6 more on their colab.
At the end of today I'm leaving this in a working-but-incomplete state: I've added all tasks I could find, updated the Colab, but am not yet happy yet.
REGION_TO_CATEGORY
and REGION_TO_DESCRIPTION
boxes are drawn incorrectly.REFERRING_EXPRESSION_SEGMENTATION
, I'm getting one large [[[polygon]]]
, regardless of what inputs I try.Planning to dig deeper, simplify and add unit tests tomorrow.
Hi @SkalskiP,
This now has:
Where it's lacking is:
["", "", "", ""]
are returned. My implementation ignores those.REFERRING_EXPRESSION_SEGMENTATION
, I always get [[[polygon]]]
, regardless of what image and prompt I try. I did figure out what the layers represent and left some comments.I invite to look at the Colab and play around with the results.
Description
This PR adds Florence 2 support for supervision.
Run it with
transfomers
and parse withfrom_lmm(sv.LMM.FLORENCE_2, result)
.No extra documentation was added to
from_lmm
at this point.Type of change
How has this change been tested, please provide a testcase or example of how you tested the change?
Usage and testing in Colab: https://colab.research.google.com/drive/19XoQZ6LRdTUysAHUp-3JsfT_67T0rffE#scrollTo=CwPzmI9wOSQ6
Any specific deployment considerations
Run the Colab in High-RAM mode.
Docs