svonidze
commented
2 years ago Similar problem with Samsung A23 https://github.com/bharat-biradar/Google-Ml-Kit-plugin/issues/285
Closed svonidze closed 1 year ago
svonidze
commented
2 years ago Similar problem with Samsung A23 https://github.com/bharat-biradar/Google-Ml-Kit-plugin/issues/285
fbernaly
commented
2 years ago This seems to be related to my comment on issue https://github.com/bharat-biradar/Google-Ml-Kit-plugin/issues/285
It is related to the output coming from the camera plugin.
LeeLoHoon
commented
2 years ago Similar problem with Samsung S20
dragongesa
commented
2 years ago @fbernaly So how to fix this issue? any suggestion?
is that because the resolution output coming from camera that give us the error ?
acoutts
commented
1 year ago This sounds related to my issue here: https://github.com/flutter/flutter/issues/118350
svonidze
commented
1 year ago this helped https://github.com/flutter/packages/pull/3277 !
acoutts
commented
1 year ago Glad it worked! The issue ended up being nobody was implementing YUV to NV21 correctly. The 'concatenatePlanes' approach just doesn't cut it for all cases.
fbernaly
commented
1 year ago @acoutts , @svonidze : what was the fix for this?
ghost
commented
1 year ago Request streaming in NV21 format from the camera and pass that to the mlkit plugin.
fbernaly
commented
1 year ago Do we need to update anything in our plugin?
ghost
commented
1 year ago Do we need to update anything in our plugin?
Bump the example app to the latest version of the camera containing NV21 (0.10.5)
Modify the camera controller constructor to stream NV21 on android. This way on both platforms, there will be just a single-plane frame coming back from the native side. On iOS, bgra8888 is already single-plane and appropriate to send directly into MLKit. On android, NV21 is the same.
Future _startLiveFeed() async {
final camera = cameras[_cameraIndex];
_controller = CameraController(
camera,
ResolutionPreset.high,
enableAudio: false,
inputImageFormat: defaultTargetPlatform == TargetPlatform.android
? InputImageFormat.nv21
: InputImageFormat.bgra8888,
);
// ...Future _processCameraImage(CameraImage image) async {
final bytes = cameraImage.planes.first.bytes;
// ...Maybe add some checks that the plane length is 1 but for bgra8888 and NV21 it will always be one plane.
Of course people are still free to stream in YUV420 (the default format) and do the conversion themselves, but it needs to be noted that this routine to concatenate the planes is not a working solution and will not work when the device camera includes padding in the image plane data like the devices mentioned in this issue.
What will happen is the bytesPerRow will not equal the frame width and the resulting concatenated image won't be a valid frame.
final WriteBuffer allBytes = WriteBuffer();
for (final Plane plane in image.planes) {
allBytes.putUint8List(plane.bytes);
}
final bytes = allBytes.done().buffer.asUint8List();You can see my examples here where I converted these frames into RGB to see what they actually look like to the MLKit plugin and it's very clear they are not valid images and is why the image recognition doesn't work on them.
https://github.com/flutter/flutter/issues/118350
The official MLKit example app by google avoids this issue by using the CameraX camera and streaming directly in NV21, which the latest camera plugin can now do using my PR that was merged.
fbernaly
commented
1 year ago Thanks @acoutts , I think this will solve most of the complains and issues reported when using this plugin. I will work on this changes ASAP.
fbernaly
commented
1 year ago Based on @acoutts-nydig comments, I am updating InputImage with some breaking changes to only support nv21 and bgra8888 from Camera plugin in this PR: https://github.com/flutter-ml/google_ml_kit_flutter/pull/454. It will be released later this week.
rraayy
commented
1 year ago Does there have any way to crop the face when the controller is set to NV21 ? Meet some problem when convert to image.
Dhruv-SalaryBox
commented
9 months ago How to use the InputImageFormat.nv21 with the following code -
final XFile? photo = await picker.pickImage( source: ImageSource.camera, preferredCameraDevice: CameraDevice.front)
What should be the right syntax? @fbernaly @ghost
The application written with Flutter, the Face Detection works ok on many other devices but does not work on Samsung Galaxy A23 Android 12.
Found this line in the Android Studio log
Google ML kit is not compatible with the processor?
Full log