ptejada / artillery-engine-socketio-v3

Socket.IO v3 engine for Artillery
Mozilla Public License 2.0
22 stars 15 forks source link

Support for socket.io auth header #11

Closed gnom1gnom closed 1 year ago

gnom1gnom commented 1 year ago

Fix for issue #8 As described in the official socket.io spec, the authorization should be performed using auth header. https://socket.io/docs/v4/client-options/#auth

Since using the extraHeaders wis not supported, I extended the socketioOpts with auth field. Also as the authentication token will be acquired using HTTP endpoint with HTTP engine, the only way to preserve this token across engine execution is to use Artillery context.vars, thus the auth attribute is initialised with this value.

jeffwillden commented 1 year ago

A bit of feedback on this PR: I agree with adding auth to socketioOpts in order to support Socket.io as described in the documentation. However, by storing the value in context.vars.auth during the before stage, you limit all socket connections to use the same auth credentials. The before declaration is only executed one time. So instead of running load tests for 50,000 different users, it would be the same user logging in 50,000 times. It would be preferable to provide an example where a different token is stored each time the scenario is run. That would allow for a load test of 50,000 different users, which is probably what people want to do most often.

I wonder if the Artillery PR 1068 might achieve this.

gnom1gnom commented 1 year ago

The PR 1068 is 2 years old. IMHO, if the Artillery team took good care of SocketIO, thay would not only merge this PR but provide support for v3 and v4 in the first place (so that this engine would not be reqired)

I agree with your comment on tokens. The most straight forward resolution would be to provide an array of tokens upfort and pop one every time a new socket session is initilized

    const socketioOpts = template(
      Object.assign({}, this.socketioOpts, {
        extraHeaders: {
          ...this.socketioOpts.extraHeaders,
          ...context.extraHeaders
        },
        auth: {
          ...context.vars.sessionTokens.pop()
        },
        // Require to force socket.io-client to set new headers on connection to different Namespace
        forceNew: true,
      }),

On the downside this would required a standardized variable name (like sessionTokens) and data structure (array). Anything more sofisticated won't work, as in the artillery/lib/launch-platform.js

    // the initial context is stringified and copied to the workers
    const contextVarsString = JSON.stringify(contextVars);

thus only standard types can be passed from the before flow.

Perhaps it would be more versatile if a unique session token was provided via a function implemented in the processor

function getSocketToken(context) {
  // some custom logic that 
  return sessionToken;
};

The name of the function would be defined in the engine config

  engines:
    socketio-v3:
      auth:
        tokenFunction: "getSocketToken"

The function would dynamicly provide the token in the following way

    let options = _.extend(
      {},
      socketioOpts, // templated
      tls
    );

    if (this.config.processor[this.socketioOpts.auth.tokenFunction]) {
      options['auth'] = {
        token: this.config.processor[this.socketioOpts.auth.tokenFunction](context)
      }
    }
ptejada commented 1 year ago

Thank you all for the PR and feedback here.

The main problem we are trying to solve is that we want to gain control over the Socket.IO connection handshake process. All current workarounds and solutions accept that the connection to the server is established before the scenario even starts and everything else will result in reconnections.

The solution can be a combination of adding a standalone connect option to facilitate establishing and configuring the connection to the server. However, we will still have the overhead of the WS connection established before the step is even initiated. For this, we can use the update from https://github.com/artilleryio/artillery/pull/1068, which removes establishing the connection in the pre-step stage.

Here are the two workflows we will want to accommodate:

  1. The current workflows with no auth requirements. In this case, the connection to the server would be established on the first emit step.
  2. The workflows where more control over the connection step is needed and the connection step will need to be part of the scenario itself. For example, the auth credentials could be pulled from a different HTTP service or from a file, or from a static list of values.

I will think about how we can streamline these two workflows.

ptejada commented 1 year ago

See updates from https://github.com/ptejada/artillery-engine-socketio-v3/issues/8#issuecomment-1565890444

The solution addresses the cases this PR was supposed to support.