flutter-ml / google_ml_kit_flutter

A flutter plugin that implements Google's standalone ML Kit
MIT License
907 stars 709 forks source link

Bug with SensorOrientation and DeviceOrientation #647

Open HalgandP opened 2 weeks ago

HalgandP commented 2 weeks ago

The CameraDescription is returning a sensorOrientation of 90 while the DeviceOrientation is returning portrait.

My Google Pixel 7 is in portrait mode. When i'm using the CameraPreview with the google_mlkit_barcode_scanning: ^0.12.0 the camera surface preview is rotated and streched while the displayed barcode is normal.

I found that in the CameraPreview there is this code that use a RotatedBox if the DeviceOrientation is in landscape or so:

int _getQuarterTurns() {
    final Map<DeviceOrientation, int> turns = <DeviceOrientation, int>{
      DeviceOrientation.portraitUp: 0,
      DeviceOrientation.landscapeRight: 1,
      DeviceOrientation.portraitDown: 2,
      DeviceOrientation.landscapeLeft: 3,
    };
    return turns[_getApplicableOrientation()]!;
  }

  DeviceOrientation _getApplicableOrientation() {
    return controller.value.isRecordingVideo
        ? controller.value.recordingOrientation!
        : (controller.value.previewPauseOrientation ??
            controller.value.lockedCaptureOrientation ??
            controller.value.deviceOrientation);
  }

I assume the CameraPreview internally use the camera.sensorOrientation to display the surface view.

As a result my camera surface view is completely stretched: image (39)

But if i change the camera preview to use the camera.sensorOrientation, everything works fine:

int _getQuarterTurns() {
    int rotateTurns = 0;
    switch (cameraDescription.sensorOrientation) {
      case 90:
        rotateTurns = 1;
      case 180:
        rotateTurns = 2;
      case 270:
        rotateTurns = 3;
    }

    return rotateTurns;
  }

image (38)

So why is there a difference between the device orientation and the sensor orientation ?

I have no problem with the native camera app of my phone.

While using camera flutter libs, i have to bake orientation of captured image if i want them to be in the right orientation.

Platform (please complete the following information):

Other code for context:

class _CameraViewState extends State<CameraView> {
  static List<CameraDescription> _cameras = [];
  CameraController? _controller;
  int _cameraIndex = -1;

  @override
  void initState() {
    super.initState();

    _initialize();
  }

  void _initialize() async {
    if (_cameras.isEmpty) {
      _cameras = await availableCameras();
    }
    for (var i = 0; i < _cameras.length; i++) {
      if (_cameras[i].lensDirection == widget.initialCameraLensDirection) {
        _cameraIndex = i;
        break;
      }
    }
    if (_cameraIndex != -1) {
      _startLiveFeed();
    }
  }

  @override
  void dispose() {
    _stopLiveFeed();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return _liveFeedBody();
  }

  Widget _liveFeedBody() {
    if (_cameras.isEmpty) return Container();
    if (_controller == null) return Container();
    if (_controller?.value.isInitialized == false) {
      return const Center(child: CircularProgressIndicator());
    }

    var cameraDescription = _cameras[_cameraIndex];

    return Container(
      color: Colors.black,
      child: CameraPreview(
        _controller!,
        child: widget.customPaint,
      ),
    );
  }

  Future _startLiveFeed() async {
    CameraDescription camera = _cameras[_cameraIndex];

    _controller = CameraController(
      camera,
      // Set to ResolutionPreset.high. Do NOT set it to ResolutionPreset.max because for some phones does NOT work.
      ResolutionPreset.high,
      enableAudio: false,
    );
    _controller?.initialize().then((_) {
      if (!mounted) {
        return;
      }

      _controller?.lockCaptureOrientation(DeviceOrientation.portraitUp);
      _controller?.setZoomLevel(1.0);
      _controller?.setFocusMode(FocusMode.auto);
      _controller?.startImageStream(_processCameraImage).then((value) {
        if (widget.onCameraFeedReady != null) {
          widget.onCameraFeedReady!();
        }
        if (widget.onCameraLensDirectionChanged != null) {
          widget.onCameraLensDirectionChanged!(camera.lensDirection);
        }
      });
      setState(() {});
    });
  }

Thanks !