GetObject: Where is the body content (i.e. image/jpeg payload)?

[hoschult]

Dear all! Thanks for providing this great s3 node! I'm using this with s3 Object storage of Open Telekom Cloud. It works perfectly.... However, I have one issue with Get Object node: Where do I find they body payload (in my case: I want to download an image / jpg) to further manipulate the image in node red?

I tried to the following (in a function node triggered by the s3 GetObject):

Buffer.from(msg.payload.BodyAsString, "utf-8")

However, it seems to me that this is not only the payload (image / jpg data) but also the metadata included.

Thanks for helping me out!

[hoschult]

I found a change described in this thread where s3 GetObject body transformed into datatype Readable: https://github.com/aws/aws-sdk-js-v3/issues/1877

However, I'm lost in how to cast /convert the BodyAsString content to a base64 datatype (string) to further work with downloaded images (which should be in the body of the request)

[hoschult]

I think I try something which will not work. Found this: https://stackoverflow.com/questions/55486103/nodejs-convert-image-between-buffer-and-string

which tells me:

Buffer.toString() default encoding is utf8, and you can't convert from utf8 back to Buffer without breaking the image.

So the Stringify option won't work for my use case since the buffer need to be converted with base64 encoding. Back to my original question: How do I get a base64 encoded object as response (to process my pictures)?

[rristov60]

Hi @hoschult, thank you very much for the opened issue and all the effort that you put in it, you've taken a proper look much before than we were able to. I will investigate this and see how we can fix it as soon as possible. For now I would suggest to disable BodyAsString and use msg.payload.Body directly. We use the following function to convert the stream to string:

const streamToString = (stream) =>
  new Promise((resolve, reject) => {
    const chunks = [];
    stream.on("data", (chunk) => chunks.push(chunk));
    stream.on("error", reject);
    stream.on("end", () => resolve(Buffer.concat(chunks).toString("utf8")));
  });

which can be found here. I'll get back at you once I have useful updates. One thing I noticed is that we iterate over the actual stream when converting it to string, so essentially when BodyAsString is enabled, the msg.payload.Body is not usable. I'll open a separate issue for this, but fixing this will come after taking care of this issue.

[rristov60]

Thank you @hoschult for the PR as well. Looks like you have figured it out, there are a few things that need to be tweaked in the code before a merge can be done, but I will leave comments about that directly on the PR.