search

vietanhdev / samexporter

Export Segment Anything Models to ONNX
https://pypi.org/project/samexporter/
MIT License
255 stars 29 forks source link

Why did you split sam into two different submodular? #12

Closed kuazhangxiaoai closed 5 months ago

kuazhangxiaoai commented 9 months ago

Why did you split sam into two different submodular named encoder and decoder? What's different between decoder and encoder? Can I use SAM as single model?

Thanks for your answers.

kknk765 commented 9 months ago

It is actually really nice that SAM is split in 2. The encoder creates a representation of the image (basically a float array). This part is very slow. The decoder takes that representation and e.g. a point or a box and outputs a mask. This part is fast. So the great thing about having the encoder and decoder separately is that you can do the slow encoding once and then create multiple masks quickly by running the decoder several times with the output from the encoding but with different point/box input.

vietanhdev commented 9 months ago

@kknk765 Nice explaination. Thanks!! 👍👍👍