Extended ASCII chars are sent in UTF-8 on serial port

Most of realtime commands in Grbl belong to Extended ASCII table. For example : Increase feed rate of 10% is 0x91. On the web application when we press the +10% feed rate button, the Grbl widget create the following websocket frame : send /dev/ttyUSB0 \x91\n. Before transmission on websocket, the string is translated into UTF-8 and becomes : send /dev/ttyUSB0 \xC2\x91\n. After reception and parsing in SPJS, the command to send on serial port becomes \u0091 (this Unicode code point is 2 bytes long). The transmission is done in serialport.go:327 :

_, err := p.portIo.Write([]byte(data.data))

Bytes translation creates 2 bytes array (\xC2\x91) directly transmitted on serial port. So all the realtime commands that belong to extended ASCII table are prefixed with \xC2 on serial transmission. Note : Is-it dangerous ? No, according sources, \xC2 byte is not a valid command according Grbl protocol, so this byte is discarded on reception (not stored on buffer).

Suggestion of correction (Note : I'm not a Go developer, I discovered this language two days ago, so critics are welcome). Replace serialport.go:327 with the following block :

        r := []rune(data.data)
        a := make([]byte, len(r))
        for idx, element := range r {
            a[idx] = byte(element)
        }
        _, err := p.portIo.Write(a) // n2, err :=

With this modification, realtime commands are now transmitted as single byte.

What do you think ?

SPJS is used well beyond Grbl, so changing code in serialport.go is a dangerous area because it could break all other use cases if not done correctly. I'm not sure what your code really does? It seems like you are creating a new array and copying the bytes into it, so what are you achieving with it? i.e. what does rune() do?

On Wed, Feb 7, 2018 at 10:56 AM, setaperlacloche notifications@github.com wrote:

Most of realtime commands in Grbl belong to Extended ASCII table. For example : Increase feed rate of 10% is 0x91. On the web application when we press the +10% feed rate button, the Grbl widget create the following websocket frame : send /dev/ttyUSB0 \x91\n. Before transmission on websocket, the string is translated into UTF-8 and becomes : send /dev/ttyUSB0 \xC2\x91\n. After reception and parsing in SPJS, the command to send on serial port becomes \u0091 (this Unicode code point is 2 bytes long). The transmission is done in serialport.go:327 :

_, err := p.portIo.Write([]byte(data.data))

Bytes translation creates 2 bytes array (\xC2\x91) directly transmitted on serial port. So all the realtime commands that belong to extended ASCII table are prefixed with \xC2 on serial transmission. Note : Is-it dangerous ? No, according sources, \xC2 byte is not a valid command according Grbl protocol, so this byte is discarded on reception (not stored on buffer).

Suggestion of correction (Note : I'm not a Go developer, I discovered this language two days ago, so critics are welcome). Replace serialport.go:327 with the following block :
    r := []rune(data.data)
    a := make([]byte, len(r))
    for idx, element := range r {
        a[idx] = byte(element)
    }
    _, err := p.portIo.Write(a) // n2, err :=
With this modification, realtime commands are now transmitted as single byte.

What do you think ?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/chilipeppr/serial-port-json-server/issues/43, or mute the thread https://github.com/notifications/unsubscribe-auth/AHidbT8OjH5yRpsgH9SdesamAECYT8aMks5tSfHxgaJpZM4R9N4T .

rune() extract each Unicode code point from string (As I'm a newbie in Go I refer to https://stackoverflow.com/questions/19310700/what-is-a-rune)

About my suggestion of modification, try the following code to see it's impact :

s := "\u0093"    // +10% on FR command
fmt.Printf("Current version sent to serial port this : % x\n", []byte(s))

r := []rune(s)
a := make([]byte, len(r))
for idx, element := range r {
  a[idx] = byte(element)
}
fmt.Printf("Modified version sent to serial port this : % x\n", a)

The first stream contains UTF-8 escape sequence, the second one stream is correct.

To say it with other words (and after reading the excellent description in https://blog.golang.org/strings) :

String in Golang are just and only an array of bytes (Documentation : "It's important to state right up front that a string holds arbitrary bytes"). For example len function returns byte count of a string not char count (len("ñ") == 2). So a string can hold any kind of encoding (UTF-8, UTF-32, Extended ASCII, ...).
Commands are sent to SPJS using Websocket protocole. Websocket use UTF-8 for text encoding ("Broadly speaking, there are types for textual data (which is interpreted as UTF-8 [RFC3629] text)". Source https://tools.ietf.org/html/rfc6455 Page 6).
So command strings are encoded in UTF-8 in SPJS as they arrived throw a Websocket.
[]byte(of a string variable) makes only a basic copy of string bytes in an array without any translation. But this array of bytes can't be transmitted directly to Grbl controller as it's not UTF-8 compatible.
[]rune(of a string variable) extracts Unicode Code Point (=character) of the string in a rune (=alias of type int32) array. This action permits to retrieve the original Extended ASCII code sent by web browser. The loop is just here to cast runes in bytes as each rune code is lower to 255.

I hope this description can help.

Ok, so the key i'm seeing is the way SPJS is sending strings today is UTF-8 and you're hoping for just extended ASCII. UTF-8 is 2 bytes long and Extended ASCII is 1 byte long.

The problem I think is that if a serial device expects UTF-8 to get passed through, which is how it works today, you'd break those implementations. Therefore I think it's safer to put this in the grblbuffer.go file so that this is specific to Grbl, not all of SPJS.

On Wed, Feb 7, 2018 at 11:42 PM, setaperlacloche notifications@github.com wrote:

To say it with other words (and after reading the excellent description in https://blog.golang.org/strings) :

String in Golang are just and only an array of bytes (Documentation : "It's important to state right up front that a string holds arbitrary bytes"). For example len function returns byte count of a string not char count (len("ñ") == 2). So a string can hold any kind of encoding (UTF-8, UTF-32, Extended ASCII, ...).

Commands are sent to SPJS using Websocket protocole. Websocket use UTF-8 for text encoding ("Broadly speaking, there are types for textual data (which is interpreted as UTF-8 [RFC3629] text)". Source https://tools.ietf.org/html/rfc6455 https://tools.ietf.org/html/rfc6455 Page 6).

So command strings are encoded in UTF-8 in SPJS as they arrived throw a Websocket.

[]byte(of a string variable) makes only a basic copy of string bytes in an array without any translation. But this array of bytes can't be transmitted directly to Grbl controller as it's not UTF-8 compatible.

[]rune(of a string variable) extracts Unicode Code Point (=character) of the string in a rune (=alias of type int32) array. This action permits to retrieve the original Extended ASCII code sent by web browser. The loop is just here to cast runes in bytes as each rune code is lower to 255.

I hope this description can help.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/chilipeppr/serial-port-json-server/issues/43#issuecomment-364027918, or mute the thread https://github.com/notifications/unsubscribe-auth/AHidbYGMVoERpnnIrIpw37eChak7TISAks5tSqVzgaJpZM4R9N4T .

chilipeppr / serial-port-json-server

Extended ASCII chars are sent in UTF-8 on serial port #43