googlesamples / mlkit

A collection of sample apps to demonstrate how to use Google's ML Kit APIs on Android and iOS
Apache License 2.0
3.57k stars 2.93k forks source link

onTouchListener #247

Closed AxesandGrinds closed 2 years ago

AxesandGrinds commented 3 years ago

It would be amazing if we can get an onTouchListener for the GraphicOverlay or it's list of graphics. I need a way to click on the text of a graphic and get that text string.

I need that in Kotlin and I have no clue how to do it. Can you give some advice on what I can do?

cs-googler commented 3 years ago

After you get the bounding box from the text recognizer API, you can draw a custom view within the bounding box and then implement the OnClickListener for it.

xxxView.setOnClickListener(object : OnClickListener() {
                                        fun onClick(v: View?) {
                                          // show text result
                                        }
                                    })
AxesandGrinds commented 3 years ago

Hi @cs-googler,

Thank you for your response today. Here is my function below. I am already using an onTouchListener and I am capturing touch events quite well and accurately. The problem is, when I read the text from the TextBlock, it gives me all the recognized text on screen. From my tap, it cannot differentiate what I tapped on. I think this is because the graphic overlay's graphic (TextGraphic). It seems that the way the graphic overlay is coded, all recognized text on screen is in every TextGraphic, rather than each TextGraphic belonging to a particular recognized text. If this is the situation, do you have an advice on how I get target the actual Text that is tapped on? I was using this as a starter point.

`private fun onTap(rawX: Float, rawY: Float): Boolean {

// Find tap point in preview frame coordinates.
val location = IntArray(2)
graphicOverlay?.getLocationOnScreen(location)

val x: Float = (rawX - location[0]) / graphicOverlay!!.scaleFactor
val y: Float = (rawY - location[1]) / graphicOverlay!!.scaleFactor

// Find the barcode whose center is closest to the tapped point.
var best: Text? = null
var bestDistance = Float.MAX_VALUE

for (graphic in graphicOverlay!!.graphics) {

  if (graphic is TextGraphic) {

    val text: Text = graphic.clientText

    for (textBlock in text.textBlocks) {

      if (textBlock.boundingBox!!.contains(x.toInt(), y.toInt())) {

        // Exact hit, no need to keep looking.
        best = text
        break

      }

      val dx: Float = x - textBlock.boundingBox!!.centerX()
      val dy: Float = y - textBlock.boundingBox!!.centerY()

      val distance = dx * dx + dy * dy // actually squared distance

      if (distance < bestDistance) {
        best = text
        bestDistance = distance
      }

    }

  }

}

if (best != null) {

  bestCodeCaptured(best)

  return true

}

return false

}`

cs-googler commented 3 years ago

Have you tried to draw the bounding box in different GraphicOverlays instead of adding them to the same one(view)?

jackqdyulei commented 2 years ago

Close because of inactivity