very slow: stringify body flag with base64 encoding

[hoschult]

Hi, thanks for providing the new version with stringified body in base64 encoding. Tonight I checked the functionality and at least in my runtime this is terribly slow in response..... On the other side: utf8 encoded body works like charmed in terms of performance.

• Download of 3 file per 10MB with Stringify (UTF-8): 1-2 seconds
• Download the same 3 files with stringily base64 encoding: approx 60 seconds (or even longer)

Do you have any ideas why base64 encoding is that slow in processing?

[hoschult]

I dig further into it and got the impression that the thread and/or the socket get stuck when using base64 encoding. Situation get even worse (in terms of performance) if you request several files to be downloaded in parallel.

It's no bandwidth issue (up/downstream of node red server tested with approx 50mb/s throughput) and also not an load issue (all cpus are in idle mode)

[rristov60]

Hi @hoschult, I did some investigation by myself as well, I still haven't tried out the parallel downloads, but speaking purely utf-8 vs base64 encoding it is safe to say that performance differences are expected because they are used for different purposes and they perform different operations during the encoding process. Here is a quick sum up for this:

• utf-8

  • UTF-8 encoding is a variable-width character encoding that represents characters using 8-bit code units (1 to 4 bytes per character). It is designed to efficiently handle text in a wide range of languages, allowing it to be more straightforward and faster to process when converting a buffer to a string.

• base64

  • Base64 encoding is specifically design to encode binary-to-text. It takes binary data and represents it as an ASCII string using a set of 64 printable characters (hence the name "Base64"). The process of Base64 encoding involves dividing the binary data into 6-bit groups (64 possible values), each of which is then mapped to a specific ASCII character based on a predefined table. These 6-bit groups are packed together to form a string of Base64 characters.

As for the thread and/or socket being stuck, please open a separate issue, just something to keep in mind is that this might be Node-RED specific and that tweaks about this on the node itself might not affect it. Although I believe it is a good feature to have GetObjects node which will allow the user to fetch a list of objects.

[rristov60]

Hi @hoschult, since there is no new activity on the issue, I will close it. Feel free to reopen this issue if the above comment hasn't answered your question in a precise manner or if there is still some unclarity left. That being said, feel free to open a new issue at any time!