idea: WaaS

[boneskull]

This might be far-fetched. What if Wallaby provided a cloud service which would run tests instantaneously? Kind of like instant CI.

A couple advantages:

• Leveraging something like Floobits, you could remotely pair up and see test results as you work.
• The ability of a browser to elegantly display test results is greater than that of an IDE, since most of the screen won't be a text editor.

Immediately known problems:

• Performance
• Custom setups might be difficult to reproduce
• Sandboxing/security
• Many more

However! This doesn't have to run in the cloud. Wallaby could show all sorts great info if it launched your browser and streamed data while you were working (a la Karma, except with an actual GUI). Real-time graphs, better visual feedback. It could help address some of the problems I mentioned in #674 by displaying several "contexts" at once.

[ArtemGovorov]

Initially wallaby.js was planned as a cloud based runner. After a few experiments and analysis, it was decided to start with a client side tool, but to write in in way that it may be possible to host in a cloud in future.

We have cut a few corners since then (like using locally installed node modules and not re-installing or copying them to wallaby cache), but managed to more or less keep the possible option to host wallaby core anywhere, not just on the developer's machine.

Building a cloud infrastructure to host wallaby is clearly a large chunk of work, with all the known problems you have listed and more.

Building a nicer UI for displaying various test results is a problem that can be tackled independently (as you also mentioned). And as a matter of fact, we are doing it right now - there'll be multiple tabs apart from the Test Explorer, like Coverage, Config, possibly license management, etc.

Real-time graphs, better visual feedback.

Could you please share what kind of graphs and visual feedback you'd like to see?

[boneskull]

Could you please share what kind of graphs and visual feedback you'd like to see?

Just brainstorming here.

• Test coverage as a pie. Then, you could drill down to sets of instrumented files or individual files. You'd need a way to define a "group", but this could naively be inferred by directory structure. Some developers will store their tests alongside their sources, however.
• Temporal information; "coverage-over-time" may be helpful.
• Warnings if the coverage drops below a certain threshold
• Test failures as a pie; "failures-over-time"
• If I'm being honest, configuring Wallaby can be a challenge. A UI to assist--or at the very least, a wizard--it would be helpful.
• If your tests are running in a browser, potentially having the ability to attach to Chrome Dev Tools' debugger (in general, I'd like to somehow set breakpoints for wallaby, but that's a separate issue)

[ArtemGovorov]

@boneskull Thanks for sharing the ideas! I like them all, please keep posting if you have more over time.

Configuration assistant/wizard, plus something to increase the transparency of what wallaby.js is doing behind the scenes (like displaying compilation/preprocessing steps and intermediate results) is definitely something that I think we should prioritise.

Proper debugger support is also on the roadmap, though I think it's something that would be much better if implemented right inside the editor.

[fvanwijk]

If I'm being honest, configuring Wallaby can be a challenge. A UI to assist--or at the very least, a wizard--it would be helpful.

Yeah I want wallaby-cli! :D

[splintor]

+1 for Test coverage as a pie. I'd also consider test timing as a pie, so we can easily see what takes most of the testing time (with drill-down abilities)

[ivankolev]

+1 for:

Temporal information; "coverage-over-time" may be helpful.

we have a build script that fails the build if you trend down from previous build, it will help immensely if a time-series is displayed, even a simple line plot. BTW you should open-source the wallaby.app guys, that way the community can help with a lot of those enhancements. It's not like it's your core product anyway...

[ArtemGovorov]

@ivankolev Thanks for your feedback!

BTW you should open-source the wallaby.app guys

We are discussing this possibility internally.