Add Base64 encoding for terminal communications

[zfi]

The BlockyProp browser code is expecting base64 encoding for terminal communication as of BPC version 0.7.0.

Update the client to transmit and receive base64 encoded characters to the terminal. Once that is completed, update application version number to 0.7.0 or greater to enable the feature in the client browser code.

[MatzElectronics]

And receive base64 chars and decode them.

[zfi]

@PropGit Can you fit this into you build schedule soon? We want to wrap up the loose ends on the BlockyProp constellation of projects prior to the beginning of the fall semester.

[PropGit]

@MatzElectronics @zfi - I'm having problems with this. I think I'm encoding and transmitting the data properly, but it fails to appear on the terminal and the web client console complains

Uncaught DOMException: Failed to execute 'atob' on 'Window': The string to be decoded is not correctly encoded.

when it tries to do c_buf = atob(e.data);

I've reverted to a hard-coded test with these python lines:

data = base64.b64encode('your name'.encode('ascii'))
socket.send(data)

as well as other incarnations, and I can see that the web client receives eW91ciBuYW1l but it fails to decode it.

On the console, I've also tried data = btoa('your name') and it sets data to "eW91ciBuYW1l" and doing data = atob(data) outputs "your name", so there's something else baffling me that's happening with the received data or data structure.

Maybe the data appears as a string to me but is coming across as bytes (or vice-versa, or utf-8 or something) and atob() can't handle it? Been searching but haven't found the answer.

[MatzElectronics]

Not 100% sure what's happening, but have you tried: c_buf = atob((e.data).toString());

???

Matthew Matz | STEM/Robotics Educator Parallax Inc. | Direct: 916-625-3019 | www.parallax.com | @M atzElectronics http://twitter.com/MatzElectronics

On Tue, Dec 12, 2017 at 10:42 AM, Parallax Git Administrator < notifications@github.com> wrote:

@MatzElectronics https://github.com/matzelectronics @zfi https://github.com/zfi - I'm having problems with this. I think I'm encoding and transmitting the data properly, but it fails to appear on the terminal and the web client console complains

Uncaught DOMException: Failed to execute 'atob' on 'Window': The string to be decoded is not correctly encoded.

when it tries to do c_buf = atob(e.data);

I've reverted to a hard-coded test with these python lines:

data = base64.b64encode('your name'.encode('ascii')) socket.send(data)

as well as other incarnations, and I can see that the web client receives eW91ciBuYW1l but it fails to decode it.

On the console, I've also tried data = btoa('your name') and it sets data to "eW91ciBuYW1l" and doing data = atob(data) outputs "your name", so there's something else baffling me that's happening with the received data or data structure.

Maybe the data appears as a string to me but is coming across as bytes (or vice-versa, or utf-8 or something) and atob() can't handle it? Been searching but haven't found the answer.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/parallaxinc/BlocklyPropClient/issues/99#issuecomment-351148094, or mute the thread https://github.com/notifications/unsubscribe-auth/AS0quHjS85a-aYYoxgcL5-V7RncObok9ks5s_skWgaJpZM4OuMi2 .

[MatzElectronics]

Or, maybe:

c_buf = e.data ? atob((e.data).toString) : '';

Matthew Matz | STEM/Robotics Educator Parallax Inc. | Direct: 916-625-3019 | www.parallax.com | @M atzElectronics http://twitter.com/MatzElectronics

On Tue, Dec 12, 2017 at 11:26 AM, Matt Matz mmatz@parallax.com wrote:

Not 100% sure what's happening, but have you tried: c_buf = atob((e.data).toString());

???

Matthew Matz | STEM/Robotics Educator Parallax Inc. | Direct: 916-625-3019 <(916)%20625-3019> | www.parallax.com | @MatzElectronics http://twitter.com/MatzElectronics

On Tue, Dec 12, 2017 at 10:42 AM, Parallax Git Administrator < notifications@github.com> wrote:

@MatzElectronics https://github.com/matzelectronics @zfi https://github.com/zfi - I'm having problems with this. I think I'm encoding and transmitting the data properly, but it fails to appear on the terminal and the web client console complains

Uncaught DOMException: Failed to execute 'atob' on 'Window': The string to be decoded is not correctly encoded.

when it tries to do c_buf = atob(e.data);

I've reverted to a hard-coded test with these python lines:

data = base64.b64encode('your name'.encode('ascii')) socket.send(data)

as well as other incarnations, and I can see that the web client receives eW91ciBuYW1l but it fails to decode it.

On the console, I've also tried data = btoa('your name') and it sets data to "eW91ciBuYW1l" and doing data = atob(data) outputs "your name", so there's something else baffling me that's happening with the received data or data structure.

Maybe the data appears as a string to me but is coming across as bytes (or vice-versa, or utf-8 or something) and atob() can't handle it? Been searching but haven't found the answer.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/parallaxinc/BlocklyPropClient/issues/99#issuecomment-351148094, or mute the thread https://github.com/notifications/unsubscribe-auth/AS0quHjS85a-aYYoxgcL5-V7RncObok9ks5s_skWgaJpZM4OuMi2 .

[PropGit]

It was worth a try, yes, but the .toString() addition didn't change the behavior at all. Doing console.log(e.data) just before the conversion confirms that the correct string appears; no extra or fewer, or different characters.

I'll keep looking.

[PropGit]

Using https://www.base64encode.org/ and https://www.base64decode.org/, I verified that "your name" and "eW91ciBuYW1l" match the encoding for all 8 input character sets given.

[PropGit]

Tried this article and it's suggestion to use base64.encodestring() too... no luck.

[PropGit]

I even tried converting it to a array buffer first, using this: c_buf = atob(str2ab(c_buf));

that references this code:

// Converts String to ArrayBuffer.
var str2ab = function(str, len = null) {
// Convert str to array buffer, optionally of size len
  if (!len) {
    len = str.length;
  }
  var buf = new ArrayBuffer(len);
  var bufView = new Uint8Array(buf);
  for (var i = 0; i < Math.min(len, str.length); i++) {
    bufView[i] = str.charCodeAt(i);
  }
  return buf;
};

[PropGit]

Solved it- with the help of both of you @MatzElectronics and @zfi.

atob() does indeed accept a string and the Python code was indeed encoding correctly and sending it as a string (and the .onmessage() event's resulting e.data was indeed a string type). I verified this a few different ways last night and this morning.

@MatzElectronics's emailed code got me thinking differently (thank you!) and his follow-up statement "...make sure the connection string sent right at the beginning is also base64 encoded" ultimately made me realize what was happening... the connection statement was not encoded (my mistake) and the web client code was:

1. complaining only about that connection statement transmission as being incorrectly encoded, not the many data transmissions following it (I misunderstood exactly what transmission it was choking on), and,
2. it was not displaying anything on the terminal because the non-encoded connection statement was never found, so it ignored the rest.

No need for a custom atob function... it all works properly now.

I'll verify bi-directional function too before I finalize.

[PropGit]

Fixed. Added in PR #105.