(question) tensorflow-lite model

[camhart]

My goal is have NSFW detection on android. For the time being, I'm okay with it not being super fast/efficient (though eventually I'd like to go that direction).

I converted the keras model to a tensorflow lite model using the following:

tflite_convert --inference_input_type=FLOAT --inference_type=FLOAT --output_file=nsfw_mobilenet2.224x224.tflite --keras_model_file=nsfw_mobilenet2.224x224.h5

I then followed the tensorflow-lite example provided by Google. Here's references to what I believe are the two most important files:

• https://github.com/tensorflow/examples/blob/master/lite/examples/image_classification/android/app/src/main/java/org/tensorflow/lite/examples/classification/tflite/ClassifierFloatMobileNet.java

• https://github.com/tensorflow/examples/blob/master/lite/examples/image_classification/android/app/src/main/java/org/tensorflow/lite/examples/classification/tflite/Classifier.java

I wired it up, and everything appears to run. However I'm not getting as accurate of results.

A few questions:

1. Did I miss anything when converting the model?
2. The addPixelValue method (see https://github.com/tensorflow/examples/blob/master/lite/examples/image_classification/android/app/src/main/java/org/tensorflow/lite/examples/classification/tflite/ClassifierFloatMobileNet.java) has the following:

   @Override
   protected void addPixelValue(int pixelValue) {
   imgData.putFloat((((pixelValue >> 16) & 0xFF) - IMAGE_MEAN) / IMAGE_STD);
   imgData.putFloat((((pixelValue >> 8) & 0xFF) - IMAGE_MEAN) / IMAGE_STD);
   imgData.putFloat(((pixelValue & 0xFF) - IMAGE_MEAN) / IMAGE_STD);
   }

   Where IMAGE_MEAN and IMAGE_STD are 127.5. What value should they be for this model? I've tried 127.5 and 255--both give less accurate results.

Thanks for your time

Update: Looking at https://github.com/infinitered/nsfwjs/blob/master/src/index.ts

      // Normalize the image from [0, 255] to [0, 1].
      const normalized = img
        .toFloat()
        .div(this.normalizationOffset) as tf.Tensor3D

This makes me believe I'd need to do that instead. So I believe this would be done with:

    private static final float NORMALIZE_PIXELS = 255.f;

    private static void addPixelValue(ByteBuffer buffer, int pixelValue) {
        buffer.putFloat(((pixelValue >> 16) & 0xFF) / NORMALIZE_PIXELS);
        buffer.putFloat(((pixelValue >> 8) & 0xFF) / NORMALIZE_PIXELS);
        buffer.putFloat((pixelValue & 0xFF) / NORMALIZE_PIXELS);
    }

I'm still not getting amazing results though.

[camhart]

https://www.tensorflow.org/lite/convert/cmdline_examples

Has some helpful examples, but they go a bit over my head. Anyone have any suggestions how to convert NSFWJS models to tflite? I'd err on the side of speed over accuracy (assuming the drop in accuracy isn't too significant).

This makes me think I should use the quantized model and then run something similar to the following:

tflite_convert \
  --output_file=/tmp/foo.tflite \
  --graph_def_file=/tmp/some_quantized_graph.pb \    //this would be the NSFWJS quantized model
  --inference_type=QUANTIZED_UINT8 \                       // keep this the same
  --input_arrays=input \                                                  // not sure here--would this stay the same?
  --output_arrays=MobilenetV1/Predictions/Reshape_1 \    //not sure here--would this stay the same?
  --mean_values=128 \                                                          //not sure
  --std_dev_values=127                                                         //not sure

[camhart]

Using https://lutzroeder.github.io/netron/ my guess would be input_1 may be the input_array and dense_3 would be the output_array. However when I try this I get the following:

ValueError: Invalid tensors 'dense_3' were found.

Edit: Looks like it's dense_3/Softmax

[MuhammadIbrahimSE]

Hi @camhart did you convert it successfully and integrated it with android, can you please share the file and android repo.

[TechnikEmpire]

In the latest model, there is a TFLite model already in there. You can simply open the model(s) in Netron and see what the input names are and what the output layer is.

With regards to your accuracy issue, don't apply mean substraction and ensure the scale is 255. I ran into this as well converting the model to another inference engine.

[arkty]

@camhart may be you will find this helpful. As @TechnikEmpire said, TFLite file is already included in the latest model. All you need is to configure image processor params for this model. Input has type float32[1,224,224,3],so we need to resize image and normalize it.

val image: Bitmap // your image

private val imageProcessor = ImageProcessor.Builder()
        .add(ResizeOp(224, 224, ResizeOp.ResizeMethod.NEAREST_NEIGHBOR))
        .add(NormalizeOp(0f, 255f))
        .build()

val buffer = TensorImage(DataType.FLOAT32).let {
        it.load(image)
        imageProcessor.process(it)
}.tensorBuffer

val outputs = model.process(buffer).outputFeature0AsTensorBuffer

You can find more details in project NsfwDetectHelper.kt

[colindean]

If this is still a problem, please provide the version of nsfw_detector that you're using, your Python version, your Python platform, and a code example of how you're invoking nsfw_detector, and reopen the issue.