capacitor-community / image-to-text

Capacitor Plugin for image to text processing (OCR)
Other
28 stars 10 forks source link

Recognized Text is not returning topLeft coordinate #5

Closed joptimus closed 1 year ago

joptimus commented 1 year ago

Describe the bug When using the plugin, the topLeft coordinate does not return from plugin

To Reproduce Steps to reproduce the behavior:

  1. Build the Application utilizing the provided sample code from the Sample App within the project.
  2. Launch the Application on iOS
  3. Capture a Text Image
  4. Review the Returned Data
  5. topLeft Coordinate is missing

Expected behavior The data should return coords for topLeft, topRight, bottomLeft, bottomRight

Screenshots I created a sample app where I plot the coordinates where the plugin says it found the topLeft, topRight, bottomLeft, bottomRight. In the screenshots you can see that topLeft never comes.

IMG-6577

IMG-6578

Desktop (please complete the following information):

Smartphone (please complete the following information): "@angular/common": "^15.0.0", "@ionic/angular": "7.3.0", "@ionic/cli": "^7.1.1", "@capacitor-community/image-to-text": "^5.0.1"

Additional context

Based on sample app, here is the console.log data

const photo = await Camera.getPhoto({
   quality: 90,
   allowEditing: true,
   resultType: CameraResultType.Uri,
   source: CameraSource.Camera        
});

const data: TextDetections = await Ocr.detectText({ filename: photo.path });

for (let detection of data.textDetections) {
    console.log(detection.text);
}
  "textDetections": [
    {
      "text": "A",
      "topRight": [0.43478260869565216, 0.41787439613526567],
      "bottomRight": [0.43478260869565216, 0.3285024154589372],
      "bottomLeft": [0.3671497584541063, 0.3285024154589372]
    },
    {
      "topRight": [0.6763285024154589, 0.4589371980676329],
      "bottomRight": [0.6763285024154589, 0.3623188405797102],
      "bottomLeft": [0.5845410628019324, 0.3623188405797102],
      "text": "B"
    }
  ]
}
joptimus commented 1 year ago

@dtarnawsky i think I found the issue but I do not know swift well enough to know. In your plugin.swift for iOS, there is this code to resolve the call


self.detectedText = results.map {[
                "topRight": [Double($0.topRight.x), Double($0.topRight.y)] as [Double],
                "bottomLeft": [Double($0.bottomLeft.x), Double($0.bottomLeft.y)] as [Double],
                "bottomRight": [Double($0.bottomRight.x), Double($0.bottomRight.y)] as [Double],
                "text": $0.topCandidates(1).first?.string as String?
            ]}

It looks like topLeft is missing from this

joptimus commented 1 year ago

@dtarnawsky Created a pull request that fixes this issue on iOS