[Question] How to monitor messages arriving on socket.io channel?

[gboysko]

I understand how flows can be written to emit socket.io messages and wait (for some indeterminate amount of time) for a response message, but how would we write flows that can receive arbitrary (real-time) messages sent from other virtual users?

I can envision some type of monitor function that receives messages for a given flow and can verify that they are correct. Perhaps this monitoring can occur during a "think" phase.

I don't see anything in the documentation that addresses this type of real-time message validation. Is that consistent with the vision for Artillery?

[hassy]

You're right @gboysko, there's nothing like it in Artillery yet, but it's definitely something worth adding! What is your exact use case?

One way to do it would be to define a hook (like the ones in the HTTP engine) - something like onMessage at the scenario level. Is that like what you're thinking of with monitors?

[gboysko]

Yeah, that's pretty close to what I'm thinking. My use case is what you'd expect in many chat/collaboration services: new messages arrive as users interact and I'd like to ensure that the messages received are consistent with the scenario/flow. I'd love to be able to "fail" the flow if an unexpected message was received (or an expected message contained fields indicating a failure condition).

I'm not sure how we would indicate that a flow remains in a "listening" state (other than re-purposing think).

[hassy]

Cool, this is definitely a needed feature (not just for Socket.io either, but also for WebSockets and any other async protocol we want to support like MQTT or XMPP).

As the first experiment we could go ahead with something like this:

1. Support defining an onMessage function that would be invoked for every incoming message, which you could use to implement a custom counter for example. This would be completely async.

scenarios:
  - name: "A Socket.io scenario"
    onMessage: "myOnMessageHandler"
    flow:
      - emit: "something" # flow definition here

1. Add a listen command to flows. The function specified in a listen command will receive every incoming message from that point onwards until it gives control back to the flow.

scenarios:
  - name: "A Socket.io scenario"
    flow:
      - emit: "something" # flow definition here
      - listen: "myHandlerFunction"
      - emit: "somethingElse"

I can give it a go some time in the next few weeks, but it's up for grabs if someone wants to give it a go and open a PR before then.

[gboysko]

I'm new to Artillery and the code base, but may give it a try. I like the second option better as it reflects my own use case. If I did, I think it would be helpful to clearly understand the semantics.

For example, I would suspect that the invoked handler function (myHandlerFunction) can convey failures encountered, right? And that the "listening" phase of the flow would block the following emit steps from proceeding.

[hassy]

@gboysko yep, in the HTTP engine handler functions can do one of two things:

1. Emit an error event which will log the error (in the shell output / JSON log) but won't abort the scenario.
2. Invoke the callback with an error object - that will log the error and abort the scenario.

As to whether the emits following a listen would be blocked until the handler function returns - I think that both makes intuitive sense and I am not sure how the opposite could be expressed cleanly in a flow spec. :)

[hassy]

@gboysko any luck? (if you've had the time) feel free to open a WIP PR, I'm happy to help out!

[gboysko]

Thanks, @hassy. I unfortunately, have not yet had the time. I am perhaps able to work on a different enhancement request (if you deem it worthy of pursuing [#171]). For me, this issue (#170) is more of a longer term concern, whereas #171 is somewhat nearer term.

[hassy]

@gboysko yes, #171 is certainly something very useful!

[hassy]

For when we're implementing this feature - we now have the means to attach a handler to any incoming message in Socket.io now:

https://github.com/shoreditch-ops/artillery-core/pull/192/commits/ed21d66a87f736e49ec7543a1f571fe744a5936a#diff-4c8bbeda468bf6bfdd048bd21ee2d037R248

[alvatech]

@hassy @gboysko Is anyone working on this feature? I need this feature but I'm new to the Artillery code base. Any pointer on how this can be implemented?

[ruchi-shakya]

@hassy hi! I am looking if you have tried to implement Listen/Receive Events feature on Artillery. This feature is something I need in my load testing where I am trying to Emit and Listen on like 50+ users.

Any work around from your side would be really appreciated. Thanks!!

[mcmaclachlan]

@hassy I think issue here is pretty much what I was looking for in my discussion question. Any chance it's implemented in the new version of artillery?

[simonbuehler]

the capturing of a response is there for websockets, see #917 , but the pure listening e.g. directly after connection without a message beeing sent is still missing iiuc . adding a handler for onMessage to the config.ws object in yaml also doesn't work (https://artillery.io/docs/guides/guides/ws-reference.html#Other-configuration-options and https://github.com/websockets/ws/blob/master/doc/ws.md#websocketonmessage)

[simonbuehler]

don't know if this should be a different issue but having the websocket support for capturing/listening for responses without a message sent would be awesome.

testing self hosted pusher like getting:

target: "wss://socket.example.de:6002/app/thekey?protocol=7&client=js&version=4.4.0&flash=false"

would upgrade the http to a ws connection and the initial message from the server contains a socket_id and such which needs to be captured and added to consequent ws requests.

[Dissfall]

Hi! There is any info about implementation status of this feature? Is it planning?

[andyglez96]

hey @hassy, any update of this feature? can we already listen for messages in a channel?

[AdrKacz]

Hi 👋

Any news on this feature ? I've seen this feature #917, but it allows to capture only after sending something. What if I want to send something, then wait for a particular message (the response to what I've sent) to arrive (among hundreds of others)?

If this issue is already solved, let me know 😊

[eliezercazares]

I need diz

[hoang-pham]

I need a way to listen and handle events arriving from the Socket.IO server without an emit event.