Migrate to TensorFlow.js

[mikeemoo]

I'm just posting here to register my interest in this - I know nothing about tensorflow or deeplearn so won't be able to help, but very interested in playing with the result.

[reiinakano]

TensorFlow.js is just a later version of deeplearn.js renamed. Results won't actually change (in theory), although maybe it'll be more efficient and run on more platforms, who knows.

[cvalenzuela]

This might be relevant to see: https://github.com/zaidalyafeai/Fast-Style-Transfer-Keras-TF.js

Althought the repo says:

This is done using tensorflow.js check the file fast-style.html. Note that I have and editted version of the source package tf.min.js. It containts many custom layers like corpping and upsampling that are yet to be implemented in tf.js. Make sure to use that file.

[DanRuta]

I got this working with tensorflowjs over here https://github.com/DanRuta/StyleSight, on a separate demo project.

One thing I couldn't get working the same was the GPU -> GPU output rendering, to skip sending the values through the CPU first. I couldn't find the new way to do it, if there is still one.

If it'd be helpful to anyone, I could isolate it into a separate repo, without anything unrelated, like build tools, libraries, or frameworks.

[mikeemoo]

Extracted the code from StyleSight into a simple standalone file:

https://gist.github.com/mikeemoo/b4affe711d56bd9828e8db7ce73ae13c

import { loadStyle, predict } from "./net";

const img = new Image();
img.onload = async () => {
  loadStyle("udnie").then(async style => {
    const result = predict(img, style);
    const data = await result.data();
    const [height, width] = result.shape;
    ... do whatever
});
img.src = "...jpg";

Seems extremely slow at the moment, though! Crashes my little laptop with anything bigger than a couple of hundred pixels.

[DanRuta]

Nice one! :) Yep, that happens for me, also. I've noticed GPU memory leaks on several tensorflowjs projects (not just my own), so maybe there's an issue with tfjs.

As for the speed, I think it may be a little faster with GPU -> GPU output rendering, but even so, the model is quite complex. Even with a small resolution input, I don't get more than a few frames per second, on a 1080.