Suggestion: Running multiple instances at once?

[mvestergaard]

Say you have an application with both a back-end and front-end. You have it organized in the same project.

The back-end and front-end will both need different wallaby config files to run.

I attempted to configure this as two different runners in webstorm, and running both, but it seems like the last one you start just takes over from the first one, and any file change you make affecting the first you started, causes an error in the second.

Would it be possible to either somehow allow multiple instances of wallaby to run at the same time, or maybe allow the wallaby config to be split into multiple sections that each run separately?

[ArtemGovorov]

It's a good idea, I will have a look into what it takes to implement.

[pixelprodev]

I also attempted to create two different runners in the same project. The configurations did not cause any errors between one another, but only the last one I started worked. +1 for having both run in parallel. Since this is a commercial product, having the ability to test node/io driven applications and their front ends would be a shame to do without. Keep up the good work sir.

[fvanwijk]

Haven't tested it, but could you run two instances simultaneously by setting different projectCacheDirs per config file?

[ArtemGovorov]

@fvanwijk project cache dir is now unique (a hash from the path of the config file), so it's not an issue. Issues to solve and implement in this case are in the IDE integration.

2all: If a file opened in an editor is covered by two running instances of wallaby.js, how would you expect it to display the coverage: overwrite from the latest received result, merge the coverage, have 2 columns of coverage indicators in the editor gutter, something else?

[mvestergaard]

Merge the results IMO. Maybe with an indication of which instance the error came from, so you can see if the problem is browser related for example when running the same code both server and client side.

[fvanwijk]

Agreed with @mvestergaard. In my situation there is Project A, Project B and a 'Shared' project with files for A and B. So I have 3 configs.

When I run config A and config Shared in parallel, I expect to see the aggregated coverage (union). Istanbul already does this by default since it aggregates all lcov files it can find, if I remember correctly.

[ghost]

:+1: for merging the inline coverage report for two running wallaby runners. I have a Wallaby config for client unit tests and one for server unit tests. Some files are used on the server and client side (isomorphic code).

It would already help if the coverage report for server only files wouldn't disappear when the client tests run (and vise versa).

[fvanwijk]

I thought about it a little more... I'd rather have two rows over coverage indicators so that I can see from which instance the coverage measurements are, instead of merging measurements.

[samhatoum]

If a file opened in an editor is covered by two running instances of wallaby.js, how would you expect it to display the coverage: overwrite from the latest received result, merge the coverage, have 2 columns of coverage indicators in the editor gutter, something else?

I would split the icon diagonally for 2 instances, 3-way for 3 instances and quarters for 4 instances.

I can easily foresee wanting to run 3 instances. One for client, one for server and another for cucumber (when #46 is supported)

Here's another idea for how to get more info into a small place: http://lab.ejci.net/favico.js

[joergwasmeier]

Hi,

just want to ask is there is any progress on this because iam running in the same issue with Webstorm and for now i need to switch every time my running task (Client / Server).

[ArtemGovorov]

@joergwasmeier no progress yet, but I'll update the issue when there's some.

[samhatoum]

Perhaps another trick to do this would be to have multiple environments in one configuration, rather than having multiple instances.

Something like:

  env: [{
    name: 'server':
    type: 'node',
    runner: path.resolve(process.env.HOME, '.nvm/versions/node/v6.3.0/bin/node'),
  }, {
    name: 'client':
    runner: require('phantomjs2-ext').path,
    params: {runner: '--web-security=false'}
  }],

And then scope out the config for each environment type, and even have some config that applies to both:

  files: [  // common
    'src/imports/**/*.js',
    {pattern: 'src/imports/**/*spec.js', ignore: true},
  ],
  `server`: {
    files: [  // server environment only
      {pattern: 'src/imports/*(browser|ui)*', ignore: true},
      {pattern: 'src/imports/@(browser|ui)/**/*.js', ignore: true},
    ]
  }
  `client`: {
    files: [  // client environment only
      {pattern: 'src/imports/*(server)*', ignore: true},
      {pattern: 'src/imports/@(server)/**/*.js', ignore: true},
    ]
  }

This could potentially make your life easier for the wallabyjs.com/app integration

[michelalbers]

Yip, +1 here :)

[fabiob]

Any progress on this front? All our projects need multiple test configurations (frontend/backend).

I would like to contribute with my 2¢ on this:

If a file opened in an editor is covered by two running instances of wallaby.js, how would you expect it to display the coverage: overwrite from the latest received result, merge the coverage, have 2 columns of coverage indicators in the editor gutter, something else?

• When running two instances of wallaby.js, I would expect two separated coverage columns;
• When running a single, multi-environment instance of wallaby.js (like @samhatoum suggested), I would expect the coverage to be merged.

[NeurAlch]

Would love this, I have projects with multiple configurations (frontend, backend, services, api, etc) and would be useful to be able to test them side by side, since I develop on all those fronts at the same time and switching makes it a bit more tedious. I would prefer having separation on the tests (columns, tabs, etc) since in my perspective merged coverage would make it difficult to find where is the issue (react client? node web server? api server?).

[kennethlynne]

I've fell in love with vs code workspaces, but I also love wallaby and it does not work well in this case :(

[fuzzybair]

If a file opened in an editor is covered by two running instances of wallaby.js, how would you expect it to display the coverage: overwrite from the latest received result, merge the coverage, have 2 columns of coverage indicators in the editor gutter, something else?

I would split the icon diagonally for 2 instances, 3-way for 3 instances and quarters for 4 instances.

I can easily foresee wanting to run 3 instances. One for client, one for server and another for cucumber (when #46 is supported)

Here's another idea for how to get more info into a small place: http://lab.ejci.net/favico.js

I Like the Idea of a diagonally split icon but splitting further may not be the most feasible solution. Consider a shared library used by several small front end sites in a single solution. It would be nice to have runners for all the sites as you make changes to the shared library. If you split for each runner you would need to limit the number of runners or support a dynamic split. However if you only split once to indicate that one or more runners passed and one or more runners failed, you could provide a context menu that would list the name of the runner/project and the status pass/fail. I have not built extensions to any IDE's so I don't know what they are capable of, but from a user perspective I think it would look cleaner and be easy to understand

[inithink]

+1

[Dianoga]

VSCode doesn't appear to allow running multiple Wallaby instances simultaneously so I'm also looking for a good way to have two separate runners going (frontend and backend code).

[smcenlly]

If you want to run both at the same time with different wallaby configurations, the best option for you right now is to start two versions of VS Code and have two separate wallaby configuration files (one for frontend and one for backend). To switch between wallaby configurations, you must use the command Wallaby.js: Select Configuration File before starting Wallaby.

[patroza]

Guys, it's 2021 now, can this arbitrary limit please be lifted? We should be in control of how we want to run things, not a (paid!) extension.

[smcenlly]

@patroza - unfortunately the limitation isn't arbitrary, it's an architectural one. From an implementation perspective, a lot of our code is editor-centric which means that we assume that there is a 1:1 mapping between a single code file and a wallaby instance. This means that there are various areas of complexity that aren't simple for us to solve - for example, you may have a mono-repo with front-end and back-end code and have a separate project that is shared by both; in this case, Wallaby can't correctly calculate code coverage, associated tests, etc. between separate Wallaby instances.

Can you tell us what editor you are using and what testing framework you are using for your different projects? Wallaby does support running on a mono-repo if your testing framework is the same between your different projects, so we may be able to help configure that for you.

[patroza]

@patroza - unfortunately the limitation isn't arbitrary, it's an architectural one. From an implementation perspective, a lot of our code is editor-centric which means that we assume that there is a 1:1 mapping between a single code file and a wallaby instance. This means that there are various areas of complexity that aren't simple for us to solve - for example, you may have a mono-repo with front-end and back-end code and have a separate project that is shared by both; in this case, Wallaby can't correctly calculate code coverage, associated tests, etc. between separate Wallaby instances.

I think part of my point is that what you mention to be complex problems that are hard to solve, may also actually be considered a feature. E.g if I would prefer to have a single editor with multiple projects, but running as separate projects with wallaby instances, then I think not being able to instrument and verify code coverage from other projects, is expected.

Can you tell us what editor you are using and what testing framework you are using for your different projects? Wallaby does support running on a mono-repo if your testing framework is the same between your different projects, so we may be able to help configure that for you.

VSCode with Jest and ESBuild. My problem was because I use a jest.config with esbuild instead of babel/tsc. After changing that, my problems went away, and I think the default auto configuration right now is acceptable. I'm especially sensitive to the removal of choice especially by hardware vendors, in closed source, we basically have no choice and it is very frustrating. I think in software this should not be necessary.

But - feel free to discard it as a rant :) Right now the root of my problem is resolved, so I have no need anymore for the flexibility. Appreciate the helpful response regardless!