Shim tests

[KevinPhemius]

Hello,

is there a way to quickly test the core with a "mock-shim" (or mock-backend) that just send basic messages with the NetIDE API 1.0 (like the core-client @TimVinkemeier provided)? The core.connectivity expect messages with the NetIDE API format and sending basic text messages like ['shim', 'test'] doesn't work anymore (because the lib.netip parser can't decode "test").

Best,

Kévin

[ElisaRojas]

Hi all,

@andres-beato has prepared a new client (based on Tim's) that sends NetIDE API 1.1 messages based on the Java library (he was also preparing one using the Python library, just to check both, but I don't know if this last is finished/tested).

And this is what he got (see a screenshot with the components: shim + core + logger (one component in each window)):

Therefore, we have two main issues now:

1. As you can see, the logger is receiving the messaged in two parts now. We don't know why this is going on, but initially the logger is just a simple loop listening to the socket. The LogPub code seems ok as well. But clearly something is going on in ZeroMQ (the message is being split into two different in some part in between). Any ideas? @andres-beato can share here both codes (logger and LogPub) if needed.
2. The shim needs to have as senderId the string "shim", otherwise the arriving messages will be recognised as backend's and not as shim's. I think it would be nice to have an extra function in the library (both Java's and Python's) to format that senderId as the core demmands for recognition. For example, if we choose "001" as senderId, then the library function would transform it to "shim001" (for example), and then the core should just check the first part of the senderId (changing the "equalto" sentence to "startswith"). In that way, we don't force shims and backends to know about how the core recognise who is who. This is just a style thing for future coding. But any other suggestion will also be great :)

Best,

[KevinPhemius]

Hello,

1. The messages are 0MQ multipart messages, (see sample code below) think of it as an envelop with two pieces of paper inside.

   zmq_message.add("0_" + originId);
   zmq_message.add(message.toByteRepresentation());
   [some code]
   zmq_message.send(sendSocket);
   

   This allow the separation of the origin_id (which is a String) and the data (which is a byte array).

2. As we only have one shim, the origin_id for shim messages shouldn't matter, we only need to discriminate between backends. I put "1_shim" to be consistent with "0_backenID" so that the logger could maybe parse the first character to discriminate between shim/BE and then (if BE) foud out the backendID (because 1 would always mean the shim).

Edit: To complete my previous message: The message form the shim/BE follows the NetIDE intermediate protocol (defined here https://github.com/fp7-netide/Engine/tree/master/libraries/netip). What the LogPub sends to the Logger is 1) who sent the message and 2) the message itself. We can concatenate the String+byte array in a single message or let the Logger assume that the messages from the Logger will be in two parts. Dealer's choice (I personally prefers the second option because we have a clear separation and it seems easier to debug?).

Best,

Kévin

[TimVinkemeier]

The messages are 0MQ multipart messages, (see sample code below) think of it as an envelop with two pieces of paper inside.

Exactly, didn't my Java Client handle that already?

As we only have one shim, the origin_id for shim messages shouldn't matter, we only need to discriminate between backends. I put "1_shim" to be consistent with "0_backenID" so that the logger could maybe parse the first character to discriminate between shim/BE and then (if BE) foud out the backendID (because 1 would always mean the shim).

I guess @ElisaRojas was referring to the senderId that has to be specified for the ZeroMQ connection. I do not have a strong opinion there, we can make it configurable, or say it has to contain shim - I don't care, but we have to specify it. (So in some sense the shim always has to know how to identify against the core ;))

[andres-beato]

Hi,

thanks for your answers! I didn't realize that we had to receive as a multipart messages. I changed it and now it works correctly, you can see the example below.

I have just uploaded to SVN the new client (based on Tim's) that sends NetIDE API 1.1 messages using the Java library. It is in the Playground\ZeroMQ\shim_client_java folder.

[image: Imágenes integradas 1]

Best,

2015-09-09 15:52 GMT+02:00 Tim Vinkemeier notifications@github.com:

The messages are 0MQ multipart messages, (see sample code below) think of it as an envelop with two pieces of paper inside.

Exactly, didn't my Java Client handle that already?

As we only have one shim, the origin_id for shim messages shouldn't matter, we only need to discriminate between backends. I put "1_shim" to be consistent with "0_backenID" so that the logger could maybe parse the first character to discriminate between shim/BE and then (if BE) foud out the backendID (because 1 would always mean the shim).

I guess @ElisaRojas https://github.com/ElisaRojas was referring to the senderId that has to be specified for the ZeroMQ connection. I do not have a strong opinion there, we can make it configurable, or say it has to contain shim - I don't care, but we have to specify it. (So in some sense the shim always has to know how to identify against the core ;))

— Reply to this email directly or view it on GitHub https://github.com/fp7-netide/Engine/issues/21#issuecomment-138915815.

Andrés Beato Research Engineer

Telcaria Ideas SL

Calle Barrionuevo 8, Local 1

28911 Leganés (Madrid)

Spain