facebookresearch / sam2

The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.
Apache License 2.0
12.07k stars 1.09k forks source link

convert onnx #3

Open BaiCHao12 opened 3 months ago

BaiCHao12 commented 3 months ago

Can you provide an onnx model

DAVID-Hown commented 3 months ago

how to convert model to onnx or trt?

Aimol-l commented 3 months ago

me too,i need .onnx to use onnxruntime on c++ :-)

ibaiGorordo commented 3 months ago

I have only converted the image predictor, but here is a Colab notebook with the conversion script: https://colab.research.google.com/drive/1tqdYbjmFq4PK3Di7sLONd0RkKS0hBgId?usp=sharing

And here repository to test the converted models: https://github.com/ibaiGorordo/ONNX-SAM2-Segment-Anything

bhack commented 3 months ago

https://github.com/vietanhdev/samexporter

BaiCHao12 commented 2 months ago

https://github.com/vietanhdev/samexporter

Thank you,I want to input box

bhack commented 2 months ago

There are some issues converting the video version. More details at https://github.com/facebookresearch/segment-anything-2/issues/186

RCLaplace commented 2 months ago

I am learning how to convert pt models to onnx and trt models, but I don't know where to start and how to dissect the model structure. Can anyone teach me?

Aimol-l commented 2 months ago

I am learning how to convert pt models to onnx and trt models, but I don't know where to start and how to dissect the model structure. Can anyone teach me?

.pt is essentially a dict, and you can read all the keys of the dict, which are usually the modules used. for instance: Sam_2hiera_tiny. pt contains "image_decoder", "memory_attention", "memory_decoder", "prompt_decoder",“Mask_decoder "," obj_ptr_dej "," mask_downsample ". You can export these modules as separate .pt files and export them independently as onnx, or you can directly use sam_2hiera_tiny. pt to export onnx for different modules 屏幕截图_20240828_114958 Please refer to it later https://github.com/Aimol-l/SAM2Export.git (Although memory'attention. onnx cannot be successfully exported, the export process is similar) or https://github.com/vietanhdev/samexporter

bhack commented 1 month ago

For the onnx video export see: https://github.com/axinc-ai/segment-anything-2