Working with raw byte data from binding

[Tertiush]

Hi All

I'm trying to work directly with the bytes received from a serial link. I got to the point of representing the received messages as bytes, i.e. item.state.toString.getBytes() will look like [-17, -65, -67, 20, 15, 3, 27]. My instance just has about 100 entries at a time.

The problem is now working with this byte array. I need to find the location of certain sequences inside, then extract the bytes that are in-between these sequences (start-stop headings for my data). There's no methods available to do this, as all of them are only available at the string level. Unfortunately the byte data falls well outside the normal UTF-8 encoding, so I cannot simply use something like string.substringAfterLast("X".getBytes()) to compare with, as X in this case is not defined in UTF-8.

In essence I'd like to know if there are some tool-set available from within the Eclipse IDE to assist iwth working with bytes. I've googled a lot and till date cannot event create a byte variable in Eclipse which will have the corresponding .getBytes() value of -17. Any help is very much appreciated!

[9037568]

getBytes() returns a byte array, so I don't understand your statement that you can't create a byte variable.

Your search methodology is also unclear to me. Are your start-stop headings bytes or strings?

[Tertiush]

Thanks for the reply, I think my own confusion caused the unclear question.

I need to work with bytes as my strings contain undefined character according to he UTF8 encoding that Eclipse uses.

So I convert my received string (from serial binding) to bytes using getBytes(). My issue is working with these bytes. I cannot use an IF statement, or any other conditional statements or methods as I don't know how to represent a hard coded byte in the IDE. For instance, in an IF I want to test a single byte for the value - 17 (singed byte, as eclipse represents it), but I get a lot of errors about incompatible types. Even '-17'.getBytes().get(x) doesn't work for any x, as UTF coded values over 128 required more than one byte, none of which would be a true - 17 byte.

I might be missing something fundamental here...

[9037568]

Everything depends on your original received/input string. It would help if you could provide a sample.

The string "-17" consists of the bytes 0x2D 0x31 0x37.

[Tertiush]

So I eventually got it to work, somewhat, at least getting the bytes to display correctly. The key it seems is that you can specify the encoding to use, e.g.

AlarmSerialInput.state.toString.getBytes("ASCII").get(counter)

Adding in the "ASCII" resulted in my string characters being translated to old-fashioned bytes, as per the ASCII mapping. I however have a new issue now.

The code below receives an input from the serial binding (AlarmSerialInput), which can be over 200 bytes long. This needs to be broken into smaller chucks based on a defined deliminator, which will always be the first byte received in the message. These chucks are then sent for further processing to perform actions on other items.

The problem is that I cannot construct a new byte array (which will be one smaller message) with this code as the .add method is giving an error. This is in the last code-block below. Messages will typically be between 10 and 37 bytes when deconstructed from the main input stream.

I tried using integers, rather than bytes for all varioables and arrays but run into other issues then. I suspect I'm declaring the byte variables incorrectly, but have tried everything I could find on google.

Variables:

var byte x
var byte[] newInput

Rule (when received update...):

x = AlarmSerialInput.state.toString.getBytes("ASCII").get(0)

                            newInput.add(x)

                            counter = 1

                            while (!(counter == (ParadoxEventSize-1))) {

                                while (AlarmSerialInput.state.toString.getBytes("ASCII").get(counter) == x) {

                                    counter = counter+1

                                }

                                newInput.add(AlarmSerialInput.state.toString.getBytes("ASCII").get(counter))

                                counter = counter+1

                                if (AlarmSerialInput.state.toString.getBytes("ASCII").get(counter) == x) {

                                    println("Got a start and end, sending now:")
                                    println(newInput)
                                    //Send newInput HERE!!!!!!!!!!!!!!!!!!!
                                    newInput.clear()
                                    newInput.add(x)
                                    counter = counter+1
                                }

                            }

Output from console:

Size of event received: 234
Entire String not 37 bytes or Chatter: [63, 20, 15, 4, 4, 0, 2, 2, 12, 0, 0, 0, 0, 0, 2, 77, 97, 105, 110, 32, 65, 108, 97, 114, 109, 32, 32, 32, 32, 32, 32, 12, 0, 0, 0, 0, 123, 63, 20, 15, 4, 4, 0, 2, 29, 6, 0, 0, 0, 0, 0, 1, 85, 115, 101, 114, 32, 48, 54, 32, 32, 32, 32, 32, 32, 32, 32, 32, 12, 0, 0, 0, 0, 63, 63, 20, 15, 4, 4, 0, 2, 6, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 24, 63, 20, 15, 4, 4, 0, 2, 30, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 12, 0, 0, 0, 0, 58, 63, 20, 15, 4, 4, 0, 2, 58, 6, 0, 0, 0, 0, 0, 0, 83, 107, 117, 105, 102, 100, 101, 117, 114, 32, 83, 105, 116, 107, 109, 114, 0, 0, 0, 0, 0, 63, 63, 63, 12, 0, 92, 55, 116, 63, 65, 31, 63, 4, 63, 20, 15, 4, 4, 0, 2, 58, 13, 0, 0, 0, 79, 38, 0, 83, 107, 117, 105, 102, 100, 101, 117, 114, 32, 66, 114, 97, 97, 105, 107, 0, 0, 0, 0, 0, 63]
2015-04-04 00:03:42.927 [ERROR] [o.o.c.s.ScriptExecutionThread ] - Error during the execution of rule 'Paradox Alarm Event - Raw Input': cannot invoke method public abstract boolean java.util.List.add(java.lang.Object) on null

[9037568]

The issue you're having now appears to be that your loop is running past the end of the array (counter > string length). You need to protect against this by something like:

inputString = AlarmSerialInput.state.toString
bytes = inputString.getBytes("ASCII")
x = bytes[0]
newInput.add(x)

counter = 1

while(counter < ParadoxEventSize) {

  while((bytes[counter] == x) && (counter < ParadoxEventSize) ) {
    counter = counter+1
  }

  if( counter > ParadoxEventSize ) {
    //passed end of string; handle error;
  }

  newInput.add(bytes[counter])

  counter = counter+1
  if( counter > ParadoxEventSize ) {
    //passed end of string; handle error;
  }

  if( bytes[counter] == x ) {
    println("Got a start and end, sending now:")
    println(newInput)
    //Send newInput HERE!!!!!!!!!!!!!!!!!!!
    newInput.clear()
    newInput.add(x)
    counter = counter+1
  }

}

[Tertiush]

Yes, I still need to handle that possibility, but my error appears at the second line already, even before the while-loop:

newInput.add(x)

I've tried your method of adding with bytes[0] as well, but this results on this error on the variable:

Incompatible types. Expected byte or java.lang.Byte but was byte[]

And this error on the [0]:

There is no context to infer the closure's argument types from. Consider typing the arguments or put the closures into a typed context.

My way of working with byte-arrays seems to be the problem here and I have no idea how to correct it... Thanks for the help so-far though!

[9037568]

Well in that case I think it's because newInput is null when you attempt to call newInput.add... Do you instantiate newInput in code that isn't shown here?

I did a little experiment. This code fails with the error you're seeing:

 var byte x = 1
 var byte[] newInput
 newInput.add(x)

If I then change the 2nd line like this, the error does not occur:

 var byte[] newInput = #[1,2,3]

All the Xtend documentation says you should be able to do this, but I haven't been able to get it to work:

 var byte[] newInput = newByteArrayOfSize(40)

 10:32:41.585 [ERROR] [o.o.c.s.ScriptExecutionThread :50   ] - Error during the execution of rule 'test': The name 'newByteArrayOfSize(<XNumberLiteralImpl>)' cannot be resolved to an item or type.

[9037568]

And after a bit more work, I've determined that even this doesn't work. The line I referenced as working:

var byte[] newInput = #[1,2,3]

actually fails silently and prevents the remainder of the script from running.

Grrrr.

Logged new issue #2410 for this.

[Tertiush]

Lol, I just got home and though I'd give your code a go..... Ok, so its back to google for now. Thanks a lot for the effort though, really appreciate the help!

[Tertiush]

I eventually used the python-based twisted matrix router implemented separately to solve this issue. Then using MQTT to publish the data to openhab.