Barracuda PoseNet WebGL Tutorial | Christian Mills

[utterances-bot]

Barracuda PoseNet WebGL Tutorial | Christian Mills

This post covers how to modify the Barracuda PoseNet project to run in the browser using WebGL.

https://christianjmills.com/Barracuda-PoseNet-WebGL-Tutorial/

[Msajt]

Hi again, I'm having a problem running the Build that I made following the tutorial, it starts ok but the RAM use starts increasing until reach the limit (100%), in your example the RAM and CPU are ok. In the OnDisable(), the engine.worker.Dispose() is getting an error, so I changed to engine.Dispose(), is it right, isn't it? I made a change in Player Settings -> Player -> Publishing Settings -> Decompression Fallback: true, in order to make the build run on web server

[cj-mills]

@Msajt Woops, that one is on me. I entered the code incorrectly in the blog post. It should indeed be engine.Dispose() as in the demo project.

Looks like I also forgot to actually type out the Resources.UnloadUnusedAssets(); at the end of the Update() method as well.

I'll get the blog post updated later today and check for additional errors. I was in a bit of a rush when typing that up and was not as thorough in proofreading as I should have been.

In the meantime, you can check out the WebGL branch as a reference.

[yendou23]

Hello, thanks for the tutorial and amazing work, trying to understand it. Do You think its possible to apply this to the style transfer too? so it could work in webgl?

[cj-mills]

Hi @yendou23, I will need to test if the style transfer model is compatible with Barracuda's pixel shader backend. However, I am concerned that it would not be feasible from a performance perspective, given how demanding the style transfer model is. The performance drop between the compute shader backend and the pixel shader backend can be significant. I'll post an update here when I get a chance to test it.

[yendou23]

Wow thank You for fast answer, i was hoping to use it for just one or few frames in some generative art experiments. If You happen to find a way, please let me know. fingers crossed:) Thanks again and take care. Jan

po 25. 7. 2022 v 20:37 odesílatel Christian J. Mills < @.***> napsal:

Hi @yendou23 https://github.com/yendou23, I will need to test if the style transfer model is compatible with Barracuda's pixel shader backend. However, I am concerned that it would not be feasible from a performance perspective, given how demanding the style transfer model is. The performance drop between the compute shader backend and the pixel shader backend can be significant. I'll post an update here when I get a chance to test it.

— Reply to this email directly, view it on GitHub https://github.com/cj-mills/christianjmills/issues/24#issuecomment-1194533746, or unsubscribe https://github.com/notifications/unsubscribe-auth/APDMHATIQN7G5E6DS4GCRY3VV3UG7ANCNFSM5ROZDK2A . You are receiving this because you were mentioned.Message ID: @.***>

[cj-mills]

Hi @yendou23, unfortunately, the Pixel Shader and CPU backends do not support the style transfer model. Depending on your goals, there are probably more optimal ways to generate individual stylized images in the browser.

For example, you could host the model for free on HuggingFace Spaces and create a simple web demo using Gradio. Here is a tutorial for doing this with an image classification model.

If you need Unity for your goals, you could try using a JavaScript inference library like ONNX Runtime web and interact with it in a WebGL project (Documentation, related forum thread).

[yendou23]

Thank You so much for the reply and insights, im really new to this so im searching whats possible. Definitely will go through it and try, in the end i was just curious and needed to learn more about it.

Thank You again for your great support and time. You are the best,keep it up and take care. Best regards Jan

On Tue, Jul 26, 2022, 19:27 Christian J. Mills @.***> wrote:

Hi @yendou23 https://github.com/yendou23, unfortunately, the Pixel Shader and CPU backends do not support the style transfer model. Depending on your goals, there are probably more optimal ways to generate individual stylized images in the browser.

For example, you could host the model for free on HuggingFace Spaces https://huggingface.co/spaces and create a simple web demo using Gradio https://gradio.app/. Here is a tutorial https://tmabraham.github.io/blog/gradio_hf_spaces_tutorial for doing this with an image classification model.

If you need Unity for your goals, you could try using a JavaScript inference library like ONNX Runtime web https://onnxruntime.ai/docs/tutorials/web/ and interact with it in a WebGL project (Documentation https://onnxruntime.ai/docs/tutorials/web/, related forum thread https://forum.unity.com/threads/webgl-unity-include-an-external-javascript-file-in-jslib.1246555/ ).

— Reply to this email directly, view it on GitHub https://github.com/cj-mills/christianjmills/issues/24#issuecomment-1195769294, or unsubscribe https://github.com/notifications/unsubscribe-auth/APDMHATAZSYSE2GTVIO4UITVWANWRANCNFSM5ROZDK2A . You are receiving this because you were mentioned.Message ID: @.***>