Closed xenova closed 5 months ago
Hi there! 🤗 This PR adds a bullet point for SlimSAM + transformers.js support (which was added today), allowing these models to be run directly in a browser, using a few lines of JS code. Demo video:
https://github.com/czg1225/SlimSAM/assets/26504141/1ad1840d-0304-49a1-a0fe-6f07070073c7
Demo link + source code: https://huggingface.co/spaces/Xenova/segment-anything-web Release notes: https://github.com/xenova/transformers.js/releases/tag/2.14.0
Transformers.js uses a very similar API as the python library, as can be seen with the following example code:
Example: Perform mask generation w/ Xenova/slimsam-77-uniform.
Xenova/slimsam-77-uniform
import { SamModel, AutoProcessor, RawImage } from '@xenova/transformers'; const model = await SamModel.from_pretrained('Xenova/slimsam-77-uniform'); const processor = await AutoProcessor.from_pretrained('Xenova/slimsam-77-uniform'); const img_url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/corgi.jpg'; const raw_image = await RawImage.read(img_url); const input_points = [[[340, 250]]] // 2D localization of a window const inputs = await processor(raw_image, input_points); const outputs = await model(inputs); const masks = await processor.post_process_masks(outputs.pred_masks, inputs.original_sizes, inputs.reshaped_input_sizes); console.log(masks); // [ // Tensor { // dims: [ 1, 3, 410, 614 ], // type: 'bool', // data: Uint8Array(755220) [ ... ], // size: 755220 // } // ]
Visualization of masks:
We have add the demo link in our readme, thanks!
Amazing! Will close this PR then 😇
Hi there! 🤗 This PR adds a bullet point for SlimSAM + transformers.js support (which was added today), allowing these models to be run directly in a browser, using a few lines of JS code. Demo video:
https://github.com/czg1225/SlimSAM/assets/26504141/1ad1840d-0304-49a1-a0fe-6f07070073c7
Demo link + source code: https://huggingface.co/spaces/Xenova/segment-anything-web Release notes: https://github.com/xenova/transformers.js/releases/tag/2.14.0
Transformers.js uses a very similar API as the python library, as can be seen with the following example code:
Example: Perform mask generation w/
Xenova/slimsam-77-uniform
.Visualization of masks: