search

facebookresearch / segment-anything

The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.
Apache License 2.0
47.81k stars 5.65k forks source link

How to implement demo effect #698

Open yang-junhua opened 8 months ago

yang-junhua commented 8 months ago

Demo url：https://segment-anything.com/demo

pic

Can you tell me how to achieve the image effect? Thank you so much.

heyoeyo commented 8 months ago

It would depend on how you're presenting the image (i.e. where is the UI being provided?). If you try the 'cut-out' feature on that demo page, you can directly copy the CSS that's applied to the little cut-outs that appear on the far left menu:

.sticker:hover, .sticker-select {
    filter: drop-shadow(0.25rem 0.25rem 1px #2962d9)
    drop-shadow(-0.25rem 0.25rem 1px #2962d9)
    drop-shadow(0.25rem -0.25rem 1px #2962d9)
    drop-shadow(-0.25rem -0.25rem 1px #2962d9);
}

You have to dig through the browser inspect window (i.e. hit F12 on chrome/firefox at least) a bit, but you can toggle that CSS on/off and see the effect appear/disappear. So that's one way to do it, if you're using a web-based UI.

If you wanted to do it within python, there's an explanation of how to modify one of the notebooks to get a similar-ish outline effect in issue #589.

yang-junhua commented 8 months ago

What I want is to click on the mask and see what it looks like.

pic1

Clicking the mask will generate an svg element (with the picture).Can you tell me how to generate an svg element? How the code is implemented.Thank you very much.

heyoeyo commented 8 months ago

In python, you can generate an outline from the SAM mask using:

mask_contours, _ = cv2.findContours(uint8_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

Where the uint8_mask is the mask from SAM, converted to uint8 (so dark areas have values of 0 and the segmentation has a value of 255). This gives you polygon(s) which would need to be converted to svg. There seem to be some python libraries that can make svgs, but if you're displaying in a browser, then sending the polygon data to the browser and using something like d3 to make the svg path may be more flexible.

yang-junhua commented 8 months ago

<radialGradient id="gradient" cx="0" cy="0" r="487" gradientUnits="userSpaceOnUse" gradientTransform="translate(282.25,375.5)"><stop offset="0" stop-color="white" stop-opacity="0"></stop><stop offset="0.25" stop-color="white" stop-opacity="0.7"></stop><stop offset="0.5" stop-color="white" stop-opacity="0"></stop><stop offset="0.75" stop-color="white" stop-opacity="0.7"></stop><stop offset="1" stop-color="white" stop-opacity="0"></stop><animateTransform attributeName="gradientTransform" attributeType="XML" type="scale" from="0" to="12" dur="1.5s" begin=".3s" fill="freeze" additive="sum"></animateTransform></radialGradient>

  1. How to obtain the data of attribute r and gradientTransform of radialGradient and what is the calculation method?
  2. Some SVGS do not have radialGradient element, how to determine whether to add radialGradient element? Can you tell me? Thank you very much.
heyoeyo commented 8 months ago

The radial gradient looks like it's used to generate the flashing animation that appears after creating a mask:

radial_grad_anim

The r value they use is probably just an aesthetic choice to make the animation look nice, I don't know if it's calculated from anything (although I guess it could be chosen based on the size of the segmentation).