search

tucan9389 / SemanticSegmentation-CoreML

The example project of inferencing Semantic Segementation using Core ML
https://github.com/motlabs/awesome-ml-demos-with-ios
MIT License
320 stars 32 forks source link

Do I need to transform the image to model input size? And how to add mosaic on the original image? #18

Closed magic3584 closed 10 months ago

magic3584 commented 10 months ago

Hi, sorry the this issue to ask for help.

I have followed the demo.

Screenshot 2023-11-03 at 10 39 30

The input image is 1920 1080 pixel, and the result is 513 513 array, renderView is 300 * 300 pt.

Why the renderView is rendered normal even the input is not suitable? And how can I add mosaic on the original image depend on the segmentation?

tucan9389 commented 10 months ago

Hello @magic3584,

Could you give me more about renderedView? If you refer to DrawingSegmentationView, there is resizing algorithm in the DrawingSegmentationView.

And I don't follow add mosaic. Could you elaborate more about this? If you give some example image, it would also be great to understand :)

magic3584 commented 10 months ago

Thanks for the response @tucan9389

Yes, the renderView is DrawingSegmentationView. I wanna use DeepLabV3.mlmodel (https://developer.apple.com/machine-learning/models/) to reach the goal like left corner image, which is accomplished by using Apple API VNGeneratePersonSegmentationRequest.

I have accomplished like

As u can see the background is not clear. Perhaps because the original image is 1920 1080 and the output of DeepLabV3 is 513 513.

Is there any way to make the background clear like LiveFaceDetectionAndFaceParsingViewController?

Mosaic is a style like