flutter-ml / google_ml_kit_flutter

A flutter plugin that implements Google's standalone ML Kit
MIT License
943 stars 719 forks source link

in InputImage.fromBytes, InputImageData is required while InputImage.fromFile does not require it #261

Closed anil-shrestha closed 2 years ago

anil-shrestha commented 2 years ago

In InputImage.fromBytes, InputImageData is required while InputImage.fromFile does not require it

for that to work I am writing the bytes to file and then loading InputImage.

Uint8List? imageString = base64Decode(<base64 string>); final dir = (await getTemporaryDirectory()).path; final imageFile = File('$dir/a.png')..writeAsBytesSync(imageString!); final image = InputImage.fromFile(imageFile);

How can I achieve without saving bytes to file.

Thank you.

fbernaly commented 2 years ago

How did you get the bytes in the first instance? I mean the first line: base64Decode(<base64 string>)

anil-shrestha commented 2 years ago

How did you get the bytes in the first instance? I mean the first line: base64Decode(<base64 string>)

I am getting base64 string from api response.

fbernaly commented 2 years ago

Could you be more specific?

How you reading the image? Could you share the code how you are doing that? What plugin you are using to read the image? Image_picker? Camera?

anil-shrestha commented 2 years ago

Could you be more specific?

How you reading the image? Could you share the code how you are doing that? What plugin you are using to read the image? Image_picker? Camera?

final response = await http.get(<uri>);    
final jsonObj = jsonDecode(response.body);
final base64String = jsonObj['imageString'];

Uint8List imageString = base64Decode(base64String);
final dir = (await getTemporaryDirectory()).path;
final imageFile = File('$dir/a.png')..writeAsBytesSync(imageString);
final image = InputImage.fromFile(imageFile);
fbernaly commented 2 years ago

Oh, I see, you are downloading the image from the cloud. In that case I think what you are doing is correct: you need to create a File from the url. You wont be able to use the other constructor: InputImage.fromBytes because you do not have the InputImageData which comes from camera device that is taking the photo. I was looking in stackoverflow if there is an easier way to get a File from Url, but I think you approach is correct: https://stackoverflow.com/questions/59546381/how-to-get-image-from-url-to-a-file-in-flutter I dont think there is much left to do, therefore closing this ticket.

badpaybad commented 6 months ago

I have the same question too. How to convert jpeg Uint8List into InputImage. Jpeg in Uint8List can come from any source file, api ... Cause I use google ML kits, eg: ` import 'package:google_mlkit_pose_detection/google_mlkit_pose_detection.dart';

final PoseDetector _poseDetector = PoseDetector( options: PoseDetectorOptions(mode: PoseDetectionMode.single)); var poses = await _poseDetector.processImage(inputImage); `

badpaybad commented 6 months ago

I found this one android only :

` import 'package:image/image.dart' as AndroidImage; import 'package:camera/camera.dart'; import 'package:flutter/material.dart';

InputImage convertFromJpeg(Uint8List jpegData, double w, double h) { return InputImage.fromBytes( bytes: DartImageHelper.convertJpegToYUV(jpegData), metadata: InputImageMetadata( bytesPerRow: -1, format: InputImageFormat.yuv_420_888, rotation: InputImageRotation.rotation0deg, size: Size(w, h), ) ); }

Uint8List convertJpegToYUV(Uint8List jpegData) { // Decode JPEG data into an image object AndroidImage.Image image = AndroidImage.decodeImage(jpegData)!;

// Convert the image to YUV420sp format
AndroidImage.Image convertedImage = AndroidImage.copyResize(image, width: image.width, height: image.height);
Uint8List yuvData = concatenatePlanes(convertedImage);

return yuvData;

}

Uint8List concatenatePlanes(AndroidImage.Image image) { final int uvRowStride = (image.width / 2).ceil() * 2; final int uvPixelStride = 2;

final int ySize = image.width * image.height;
final int uvSize = uvRowStride * image.height ~/ 2;

final Uint8List result = Uint8List(ySize + uvSize);

// Copy Y plane (full resolution)
for (int y = 0; y < image.height; y++) {
  final int yOffset = y * image.width;
  //final Uint8List row = image.getBytes( format: AndroidImage.Format.luminance, y: y);
  final Uint8List row = image.getBytes();
  result.setRange(yOffset, yOffset + image.width, row);
}

// Copy UV plane (half resolution)
final int uvOffset = ySize;
for (int y = 0; y < image.height; y += 2) {
  for (int x = 0; x < image.width; x += 2) {
    final int uOffset = uvOffset + (y * uvRowStride ~/ 2) + (x * uvPixelStride);
    final int vOffset = uOffset + 1;

    //final Uint8List uvRow = image.getBytes(format: img.Format.cbCr, x: x, y: y);
    final Uint8List uvRow = image.getBytes();
    result[uOffset] = uvRow[0];
    result[vOffset] = uvRow[1];
  }
}

return result;

} `

badpaybad commented 6 months ago

I tested this worked well in android ` import 'package:image/image.dart' as DartImage; InputImage convertFromJpeg(Uint8List jpegData, double w, double h) { return InputImage.fromBytes( bytes: convertImageToYUV420SP(DartImage.decodeJpg(jpegData)!), metadata: InputImageMetadata( bytesPerRow: -1, format: InputImageFormat.nv21, rotation: InputImageRotation.rotation0deg, size: Size(w, h), )); }

Uint8List convertImageToYUV420SP(DartImage.Image image) { int width = image.width; int height = image.height; final int frameSize = width height; Uint8List yuv420sp = Uint8List(frameSize 3 ~/ 2); for (int j = 0, yp = 0; j < height; j++) { for (int i = 0; i < width; i++, yp++) { var p = image.getPixel(i, j); int r = p.r.toInt(); int g = p.g.toInt(); int b = p.b.toInt(); yuv420sp[yp] = ((66 r + 129 g + 25 b + 128) >> 8) + 16; if (j % 2 == 0 && i % 2 == 0) { int uvp = frameSize + (j >> 1) width + i; yuv420sp[uvp] = ((-38 r - 74 g + 112 b + 128) >> 8) + 128; yuv420sp[uvp + 1] = ((112 r - 94 g - 18 b + 128) >> 8) + 128; } } } return yuv420sp; }

`