neural style transfer

[maylad31]

HI, I am trying to modify your example to run my neural transfer model, I am new to flutter/mobile app development. My model takes input a bgr image and returns a bgr image as output. How do I update the image selected by user with the output image. Also can you guide me about rgb to bgr conversion. I get how to resize and apply normalization. Thanks

[maylad31]

I read the code more closely. i guess rgbValues[j++] = ((intValues[i]) & 0xFF); rgbValues[j++] = ((intValues[i] >> 8) & 0xFF); rgbValues[j++] = ((intValues[i] >> 16) & 0xFF); in imageconversion.dart can be used to change order of rgb. Just need help with converting the output to image(format required by flutter) and show it.

[matheper]

I am also trying to implement the Artistic Style Transfer example.

The model has float32 Image as input. It is possible to load the image and run the _interpreter:

    _inputImage = TensorImage(TfLiteType.float32);
    _inputImage.loadImage(image);
    _inputImage = _preProcess();
    _interpreter.run(_inputImage.buffer, _outputBuffer.getBuffer());

But it failed if I try to get the image back from the outputBuffer. The operation is not supported yet.

    // with TensorImage
    TensorImage _tensorImage = TensorImage.fromTensorBuffer(_outputBuffer);
    Image _image = _tensorImage.image;
    // or ImageConversion
    Image _image = Image(384, 384);
    _image = ImageConversion.convertTensorBufferToImage(_outputBuffer, _image);

Reading the code, I noticed there is no support for converting float32 buffer back to image in TensorImage.

Probably because convertTensorBufferToImage do not support float32.

I am trying to contribute to the issue. I really appreciate it if you have any feedback on that.

[am15h]

I am truly sorry for the late reply.

Hello @maylad31, BGR input is not supported by TensorImage.

You can manually convert to BGR image and load that to TensorBuffer

    int w = image.width;
    int h = image.height;
    List<int> intValues = image.data;

    List<int> shape = [h, w, 3];
    List<int> bgrValues = List(h * w * 3);
    for (int i = 0, j = 0; i < intValues.length; i++) {
      if (intValues[i] == null) {
        print(i);
      }
      bgrValues[j++] = ((intValues[i] >> 16) & 0xFF);
      bgrValues[j++] = ((intValues[i] >> 8) & 0xFF);
      bgrValues[j++] = ((intValues[i]) & 0xFF);
    }

    buffer.loadList(bgrValues, shape: shape);

[am15h]

@matheper, @maylad31 You can get image from tensorbuffer that has BGR data using the following code,

static Image convertTensorBufferToImage(TensorBuffer buffer, Image image) {
    if (buffer.getDataType() != TfLiteType.uint8) {
      throw UnsupportedError(
        "Converting TensorBuffer of type ${buffer.getDataType()} to Image is not supported yet.",
      );
    }
    List<int> shape = buffer.getShape();
    TensorImage.checkImageTensorShape(shape);
    int h = shape[shape.length - 3];
    int w = shape[shape.length - 2];
    if (image.width != w || image.height != h) {
      throw ArgumentError(
        "Given image has different width or height ${[
          image.width,
          image.height
        ]} with the expected ones ${[w, h]}.",
      );
    }

    List<int> bgrValues = buffer.getIntList();

    for (int i = 0, j = 0, wi = 0, hi = 0; j < bgrValues.length; i++) {
      int b = bgrValues[j++];
      int g = bgrValues[j++];
      int r = bgrValues[j++];
      image.setPixelRgba(wi, hi, r, g, b);
      wi++;
      if (wi % w == 0) {
        wi = 0;
        hi++;
      }
    }

    return image;
  }

Now, probably you want to display on UI. So you can use the flutter Image Widget's Image.memory.

import 'package:image/image.dart' as imageLib; 
import 'package:flutter/material.dart';

Widget getImageWidget(imageLib.Image image){
     var bytes = PngEncoder().encodeImage(image);
     return Image.memory(bytes);
}

I hope it helps. I will add support to the library itself in the next version. Please feel free to ask further if you facing any issues :-).

[maylad31]

hi, i am sorry a bit late but thanks you guys...