Trying to call textDetection() using base64 or buffer. Not working

[cbdeveloper]

I'm having trouble with an issue similar to issue https://github.com/googleapis/nodejs-vision/issues/9

I have a base64 image string that I need to pass to vision.textDetection(), and nothing seems to work.

The issue that I've mentioned suggests the following:

---

---

Don't know if this should still be working for the most recent version, because this is what I'm doing:

const [textDetections] = await vision.textDetection(
     { image: {
         source: {
           content: imgBase64
      }
    }});

And this is what I'm getting:

{ Error: 3 INVALID_ARGUMENT: Request must specify image and features.

This is my base64 string (it seems fine):

Also tried the following:

const buffer = new Buffer(imgBase64, 'base64');
const arraybuffer  = Uint8Array.from(buffer).buffer;

const [textDetections] = await vision.textDetection({image: {
  // content: buffer           // <--- TRIED BOTH WITH BUFFER AND ARRAY BUFFER
  content: arraybuffer
}});

Nothing seems to work.

Also, the docs don't mention anything about it accepting a base64 string.

https://googleapis.dev/nodejs/vision/latest/v1.ImageAnnotatorClient.html#textDetection

---

---

Environment details

• OS: Firebase Cloud Function Node v10
• Node.js version: 10
• npm version: NA
• @google-cloud/vision version: "@google-cloud/vision": "^2.1.1",

[cbdeveloper]

Just managed to make it work, by doing the following:

From this link: https://cloud.google.com/vision/docs/base64

---

---

I thought I should do this:

const [textDetections] = await vision.textDetection(
  {
    image: {
      content: imgBase64
    }
  }
);

---

Which goes against the documentation on https://googleapis.dev/nodejs/vision/latest/v1.ImageAnnotatorClient.html#textDetection

---

Also, I had to do this:

Instead of passing the base64 string as it was generated from my canvas element, with the data:image/jpeg;base64, in the beginning of the string.

---

I had to pass this:

const result = await firebase.functions().httpsCallable("readTextFromImage")({
      image: canvasBase64.split("base64,")[1]
    });

Removing the first part that contains the MIME type and the base64, label, and sending only the actual data string.

---

I think the documentation is just wrong or isn't clear enough on this topic and should be revised, if possible.

[munkhuushmgl]

@cbdeveloper Thanks for creating this issue, and I am logging a bug internally on docs and client library. I will keep you posted on any updates.

[munkhuushmgl]

@cbdeveloper Could you share your code snippet on how you are reading file into variable?

[cbdeveloper]

Do you mean the image file I was trying to send to the API?

I was reading it on client code from a <canvas> element, and I remember that I've tried something like this:

canvas.toBlob((blob) => {
  blob.arrayBuffer().then(buffer => /* process the ArrayBuffer */);
});

Maybe I was making mistakes when trying to send the buffer over through the POST request, so I've decided to send the base64 string instead. And that's why I was also trying to convert from base64 to Buffer over on the server code.

https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toBlob https://developer.mozilla.org/en-US/docs/Web/API/Blob/arrayBuffer

Note:

My use case flow was:

• CLIENT: takes picture
• CLIENT: crops picture with react-image-crop
• CLIENT: That results into the cropped image into a <canvas>
• CLIENT: Get image from canvas and send to a Firebase callable cloud function (basically a POST request under the hood)
• SERVER: Get image from POST request and call vision API

[alexander-fenster]

@munkhuushmgl Just to double check, do you need anything from us (Client Libraries team) here? Is this only related to the documentation, or is anything actually does not work?

[munkhuushmgl]

@alexander-fenster I think it's related to Docs --> https://googleapis.dev/nodejs/vision/latest/v1.ImageAnnotatorClient.html#textDetection Is is possilbe to update generated docs with static values?

[hkdevandla]

@alexander-fenster , any updates on this? Thanks!