Installing / adding to project

[royherma]

First of all @Volcomix , thanks for this awesome lib and demo!

My main questions are in terms of implementation - I've forked over the project and started integrating it with mine, as my end goal is to incorporate some of this BG removal functionality in my project.

I haven't had any experience with docker before, so my question is if its required to set up a docker project with the TFLite models in order to get this to work?

Also, in general - what would be the best way to integrate this into an existing project (vs cloning and running locally as a separate one).

Thanks!

[Volcomix]

Hi @royherma and thank you!

As of today there is no easy way to take a whole "pipeline", integrate it into an existing project and keep it in sync with this demo. At some point I was considering providing some features in an NPM package but I'm not sure how to provide an API which:

• is not too generic so that adding custom processing would be easy enough
• is not too specific so that it fits more than a single use case and is customizable enough
• doesn't overlap too much with other existing libraries (e.g. Mediapipe and TFJS which is now able to handle TFLite models in alpha)

That being said, you could have different options depending on your needs:

• You can copy the public/tflite directory and consume the js files directly in your project. You can find some details about its interface in src/core/hooks/useTFLite.ts. Some software/projects are already doing this and frequently update their copy of this directory from this repo. I'm trying to keep it up to date against tensorflow repository. Then for now, you need to implement/copy the pipelines and to maintain them manually.
• You can take a look at Mediapipe which provides a simple API to consume the 256x144 of ML Kit model. I don't have much experience with Mediapipe but as far as I understand, it should be quite easy to create a processing pipeline with their tool and to integrate into your app.
• You can try using the TFLite API of TFJS directly with GMeet model. You'll still also need to implement the post-processing by yourself.

I could be willing to reconsider providing an NPM package but I'll need some help to define what should go inside and the API.