AI portrait generator with JavaScript using TensorFlow

[vipertechofficial]

Hello everyone,

In the use case of providing an extra layer of anonymisation of selfies for a pixel art editor (which is named https://pixa.pics/ and is open-source) I am looking for a way to empower its feature on the behalf of recent AI advancements, as an open-source enthusiast, I am looking to try Nvidia's "https://github.com/NVlabs/stylegan2-ada" project out that is available as a neural model for Tensorflow in order to transfer some data representing someone's face into another representation, I think Tensorflow for the web is style available on GPU using WebGL, or throughout WebAssembly or simple optimized plain JS, so therefore, could anyone let me know how to build everything required to create a npm module that can be loaded from JavaScript which would contain trained "meta-portrait?" that could be filled with someone's appearance using web technologies?

I bet I need to play with Tensorflow Web Implementation available on NPM while packing StyleGan2 into some kind of JavScript module library that can be loaded on the web, while also train a set of "character" that will be used, if we succeed to unite all the solution's part, I will create a nice NPM module with contributor's pseudonym who wants to be listed within the readme.md file of the project.

That said, who can help me with understanding the following tasks:

• Packing a tensorflow neural model for using it with TensorflowJS?
• Using StyleGan2, does anyone have understood the documentation?
• How to train the model and store the "logic spore" for a particular portrait or character?
• Using Tensorflow implementation on WebGL, what are performance we will observe?

[mattsoulanille]

Under requirements, the model lists 12GB of video memory.

1–8 high-end NVIDIA GPUs with at least 12 GB of memory. We have done all testing and development using NVIDIA DGX-1 with 8 Tesla V100 GPUs.

You might be able to get this running on TFJS, but it'll only run on high end computers.

To convert the model to TFJS, you'll first need to save it as a TF saved model. I'm not sure how stylegan2 stores its models, but it doesn't look the same as a TF saved model. I think you'll need to inspect generate.py and find out how to get the TF model saved from it.

Once you have the TF model, you can try running tfjs converter on it. TFJS converter should give you a model that can run with TFJS, but you'll still need to reimplement the pre and post processing that the python code performs.

[gaikwadrahul8]

Hi, @vipertechofficial

I would request you to please refer above comment from @mattsoulanille and if I'm not wrong StyleGAN2 with adaptive discriminator augmentation (ADA) using .pklformat to store pre-trained model and also please go through the requirements mentioned in the official documentation of StyleGAN2 with adaptive discriminator augmentation (ADA) here and they mentioned these requirements also, 1. Linux and Windows are supported, but we recommend Linux for performance and compatibility reasons. and 2. We recommend TensorFlow 1.14, which we used for all experiments in the paper, but TensorFlow 1.15 is also supported on Linux. TensorFlow 2.x is not supported. I hope it will help you to resolve your issue. Thank you!

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you.

[google-ml-butler[bot]]

Closing as stale. Please @mention us if this needs more attention.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No