search

cucumber / aruba

Test command-line applications with Cucumber-Ruby, RSpec or Minitest.
MIT License
948 stars 161 forks source link

Add/Update project status in README #361

Closed e2 closed 8 years ago

e2 commented 8 years ago

Summary

New contributors need a sense of "direction" to know what to expect when working on the current Aruba codebase.

Expected Behavior

Normally, when I want to contribute to a project I "expect":

  1. Master to be pretty much stable and usable (or clearly explained why if not)
  2. All tests to pass locally (or clearly explained why if not)
  3. High coverage (or clearly explained why if not)
  4. Stable major releases (or clearly explained why that hasn't happened)
  5. ETAs for milestones, releases and major rework (just to get a sense of how long "wait" if needed)
  6. A sense of how "busy" things are at any given time (to decide if I should help with major work or if too many people will get in each other's way)
  7. Low friction to contributing (or good explanations why to follow conventions to the letter)
  8. A general sense of the project (to know whether to report stuff that isn't working or just "wait" for rework to finish first and stable releases to be published)
  9. Whether maintainers are invited to apply or not
  10. A general "roadmap" to align any contributions I'd make (or just drop them without starting - to avoid wasted time on feature that may no longer make sense)
  11. etc...

    Current Behavior

The biggest pain points I've had are:

  1. Waiting ages for "dependent commits" to fix a pressing issue for me (RVM support)
  2. Lots of "work in progress" without guidance on how to help push things forward
  3. Lots of constant and new deprecations before even reaching 1.x
  4. No 1.x for over a year.
  5. Deprecations messages were more a bother than useful (it would be easier to just have "recommended" documentation on how to do things instead.
  6. Aruba still uses it's own deprecations in multiple place.
  7. Not enough coverage and constantly changing internal+external API's
  8. Reading History.md is not a good way to learn what changed, since I'm already confused what is deprecated and what isn't. (And when something is, I have a hard time to understand what to do instead and especially what's "available").
  9. I'd actually prefer crashes rather than deprecations.
  10. Ruby 1.8 not dropped and purged from the codebase (wasted time I'll never get back).
  11. With all the deprecations, moving around the codebase is a bit discouraging.
  12. Hard to work out how to effectively debug and "get stuff done" with Aruba - I can never remember (or find the docs) on how to enable announcing, how to set in-process Aruba, how to set/unset variables, how to capture stderr/stdout for a given process, whether to use "run" or "run_simple", how to set things up with RSpec (config, hooks), how to actually track the execution and what's going on "under the hood" (especially when most errors are "silent" or "output wasn't matched").
  13. There's "so much" API for managing processes, I can't seem to figure out what the flow is (to debug an issue - or simply to just understand what I'm doing wrong).

    Possible Solution

Again, general "info" about the project would help. Status, plans, ongoing work, etc. If I "step in" only to have to wait about a year for "other parts of the codebase to get sorted out first" - and then I find that over the year there's still no 1.x ... and while being spammed with deprecations that don't really tell me enough to know what's available (or at least hint why something was deprecated so I could intuitively "guess" what could work instead ... all this is a bit discouraging, even if there are valid/sound/understandable reasons behind everything going on.

ghost commented 8 years ago

@e2 Thanks for your feedback. I will get back to this issue as fast as possible.

ghost commented 8 years ago

@mattwynne @aslakhellesoy I think this is a good starting point for improving developer experience. Maybe we can even use this as a base for other sub-projects.

e2 commented 8 years ago

@dg-ratiodata This guy really gets it: https://medium.com/the-javascript-collection/healthy-open-source-967fa8be7951#.lo0yohuij

aslakhellesoy commented 8 years ago

Thanks for sharing that post @e2 - I think the whole Cucumber ecosystem needs this

maxmeyer commented 8 years ago

@e2 Very good blog post. Thanks for sharing.

maxmeyer commented 8 years ago

ETAs for milestones, releases and major rework (just to get a sense of how long "wait" if needed)

That won't happen. Sorry.

Waiting ages for "dependent commits" to fix a pressing issue for me (RVM support)

RVM support is discontinued.

Lots of "work in progress" without guidance on how to help push things forward

Created issues for a lot of wip things. Some more to come

Lots of constant and new deprecations before even reaching 1.x

Don't see the problem here. Personally I prefer "deprecations" first, then remove on major version release. If you fix your lib the deprecations are gone.

No 1.x for over a year.

Yeah, that's really a problem.

Deprecations messages were more a bother than useful (it would be easier to just have "recommended" documentation on how to do things instead.

There's a lot of documentation about how to do things right. Just head over to "features/".

Aruba still uses it's own deprecations in multiple place.

Can you point me to the places where you found that stuff?

Not enough coverage

Do you talk about code coverage measure via simple_cov? There are quite a few tests in features/ and spec/. Yes, unfortunately quite a bit is uncovered.

and constantly changing internal+external API's

This will get better with aruba-1.0.0

Reading History.md is not a good way to learn what changed, since I'm already confused what is deprecated and what isn't. (And when something is, I have a hard time to understand what to do instead and especially what's "available").

What about our API docs http://www.rubydoc.info/github/cucumber/aruba/master/frames? Do those help?

I'd actually prefer crashes rather than deprecations.

Having deprecations and then removable will not change.

Ruby 1.8 not dropped and purged from the codebase (wasted time I'll never get back).

Fixed

With all the deprecations, moving around the codebase is a bit discouraging.

Yes, you're right. I hope we will get rid of those as fast as possible.

Hard to work out how to effectively debug and "get stuff done" with Aruba - I can never remember

  • (or find the docs) on how to enable announcing

Tracked by #391

*, how to set in-process Aruba, What about this? https://github.com/cucumber/aruba/blob/master/features/steps/command/in_process.feature

What abou this and this. And also added issued to track addition for documentation for rspec in #395.

There's "so much" API for managing processes, I can't seem to figure out what the flow is (to debug an issue - or simply to just understand what I'm doing wrong).

I think this will become easier after removable of deprecated methods and the like.

Master to be pretty much stable and usable (or clearly explained why if not)

It is and has more or less always been. PRs should be green before merged - see CONTRIBUTING.md

All tests to pass locally (or clearly explained why if not)

Besides the windows things which are tracked by issues, all tests should pass locally or at least should be ignore if a condition does not apply.

High coverage (or clearly explained why if not)

Mmmh. We're also using feature tests which are no reported "properly".

Stable major releases (or clearly explained why that hasn't happened)

I think we're on a good way here.

A sense of how "busy" things are at any given time (to decide if I should help with major work or if too many people will get in each other's way)

Can you give me some more details what do you mean with this? Personally I won't give away information how busy I'm at work/free time. There's some meta information about the ongoings in the project via Github UI.

Low friction to contributing (or good explanations why to follow conventions to the letter)

I think this is done in CONTRIBUTING.md now + the new templates for github issues/PRs.

A general sense of the project (to know whether to report stuff that isn't working or just "wait" for rework to finish first and stable releases to be published)

I think with the new branch model. reporting makes sense.

Whether maintainers are invited to apply or not

What do you mean with that?

A general "roadmap" to align any contributions I'd make (or just drop them without starting - to avoid wasted time on feature that may no longer make sense) etc...

Milestones are used for that.

e2 commented 8 years ago

Personally I prefer "deprecations" first, then remove on major version release. If you fix your lib the deprecations are gone.

In practice, every time Aruba is updated, new deprecations appear. There are so many of them, I'd prefer to just "start fresh" with an API without deprecations. Otherwise every minor release of Aruba is just a ton of new deprecations to debug - with not clear "upgrade notes" to just get it all done in one go. I'd rather just upgrade to 1.x and fix things once, and never look back. If I knew this was going to happen, I would've just locked to whatever version without deprecations and waited a few years for 1.x to appear.

Otherwise, if the docs contain e.g. set_env, and that's deprecated, it's discouraging to use the docs for upgrading.

Can you point me to the places where you found that stuff?

That was months ago - I don't remember by no.

Do you talk about code coverage measure via simple_cov? There are quite a few tests in features/ and spec/. Yes, unfortunately quite a bit is uncovered.

Mostly about RSpec files for every class and public method. Cucumber isn't good for testing edge cases.

What about our API docs http://www.rubydoc.info/github/cucumber/aruba/master/frames? Do those help?

No upgrade notes. I don't want to "re-learn" Aruba every single release every few months. Also, to documentation there is outdated (set_env used, Aruba::Processes::InProcess, etc.). Also, the documentation there is based on classes that are never used by users in practice (Announcer, ArubaPath, Command, ConfigWrapper, Environment, ... - almost all of them, really). This is because Aruba is used more like a DSL, so documenting internal classes makes little sense, unless they're mapped to what's available at runtime. E.g. knowing which class set_env is defined. (hint: it's NOT in the Environment class). That documentation documents just Aruba internals, except for the start page with outdated examples.

Having deprecations and then removable will not change.

By "crashed" I meant that I'd prefer to just switch to 1.x and redo things "the new way". Here, I don't have a 1.x version to switch to. Just a 0.x with tons of deprecations I have to hunt down and debug.

What about this? https://github.com/cucumber/aruba/blob/master/features/steps/command/in_process.feature

Can you give me some more details what do you mean with this? Personally I won't give away information how busy I'm at work/free time.

It's just to get a sense. If I'm busy working myself and I need something fixed in my project, I need to decide whether I can wait a few days to get a fixed merged and released ... or if I'm better off just forking, opening and issue and releasing my own gem. I don't mind waiting 5 years for my pull request to be considered, but I'd like to know beforehand if it's going to take more than a few weeks.

I'm busy too - and I need to get stuff done. So either we can work together, or I still need to get stuff done.

What do you mean with that?

If the answer isn't a straight "yes", then I'm assuming it's "no". If you're too busy and you don't want to accept maintainers or contributions ... that's weird. It doesn't make sense to carefully debug, fix and test issues just to make a PR to a dead project.

E.g. RVM functionality is deprecated. But does it make sense to reject fixes if there's no 1.x? You can't drop it in 0.x, so you might as well fix it for now. You're either supporting old features due to a lack of a 1.x, or you're not supporting 0.x and you're releasing 1.x to fix that. Otherwise, it's effectively a dead project. And asking people to rebase after a few months is extremely discouraging. (You could've just pulled the changes before reorganizing the codebase).

Right now, I feel like I just don't want to use Aruba anymore. I feel like testing things manually will take less time. I know how to program, so I don't need to go through volumes of "training wheels" or go scuba-diving to find what I need through the wreckage of deprecated methods. I'm strongly considering just using something like: https://github.com/abates/ruby_expect with RSpec instead. Or just running stuff in Docker.

Milestones are used for that.

The big question is: why hasn't 1.x been released for so long? Why is progress so slow? It's not that I want to hold anyone accountable, it's just so I can plan things on my side. "Working together" takes some commitment to availability on both side. I can't promise I'll still even be programming in Ruby 2 years from now, so if my issue/pull request is hanging out there for weeks, I might as well not make it. Especially if I post a bug fix and releases take months, there's almost no point in "sharing" my fix, because other people will still be hit by it weeks after my PR.

Don't take it personally - I'm just already discouraged beyond salvation. I'm mentioning all this so maybe you can fix things before the next person becomes discouraged. (I know how it is not to get constructive feedback).

mattwynne commented 8 years ago

No upgrade notes.

There is this.

@e2 would you find it easier, as someone who depends on the API, if we basically just shipped a 1.0 right now and then started using SemVer to signal new API changes from then on?

e2 commented 8 years ago

@mattwynne -

There is this.

That's not really useful for users because of all the "noise". If I update Aruba on an older project I get lots of deprecations. History.md doesn't really help me get through them. If a method is renamed, that's ok. If there' something bigger changed, I want to know how to do "X", especially if there's a new way to do that.

Having tons of deprecations in a 0.x version is really weird and confusing. It's not that the API is changing that's bothering me, it's that changing my code just takes longer. As a user, I don't want to learn the internals of Aruba. I want to fix what's broken and focus on what's valuable to me and my clients.

It's a bit like I have to "relearn" upon every minor update of Aruba, because I don't have a quick "cheat-sheet" of how to avoid deprecations and use whatever is recommended instead.

Psychologically, during development, a deprecation is the same as a crash - it's hard to continue (it's an eyesore) with deprecations appearing. So really a NoMethod error with a backtrace helps "fix" deprecations much faster than browsing through a history file, or Aruba's source code.

@e2 would you find it easier, as someone who depends on the API, if we basically just shipped a 1.0 right now and then started using SemVer to signal new API changes from then on?

It depends on the roadmap and bugfix support. In practice, major version upgrades are for removing deprecated functionality, really. It doesn't make sense to have 12 release for every minor in 0.x with a bugfix, so I understand that to get bugfixes it's best for me to rely on upgrading minor versions.

So with Aruba, I'm upgrading minor versions to get bugfixes, mostly. (Because expecting them to be backported to older versions and released will be unlikely).

The problem with upgrading minor versions is that I'm getting lots of deprecations that I don't even want to investigate. And deprecations are only for people stuck with an older version of Aruba. And without a 1.x, that's me - I'm stuck with ARuba 0.x with deprecations, and the next 0.x minor upgrade will probably have more deprecations.

You also can't change the API once you get to 1.x. You can only "add" to the API and release bug fixes (as patch-level versions). You can only "change" the API in 2.x.

If Aruba really wants to "decide on all deprecations up front", what it should have is a "tool" to convert my code to not be "deprecated". If it doesn't make sense - due to effort needed to create such a tool (which I understand) - I'd like to have a quick cheat-sheet of how to upgrade my API.

I'm not a "dumb user" it's just that I don't want to waste time on something that's just a cost with no benefit. E.g. changing set_env to set_environment isn't a problem, because the name suggests what to change exactly. Changing it is just a find and replace.

A "benefit" would be having some version (1.x? 2.x?) where run just fails if it's given parameters it doesn't understand. That would've saved a lot of time and frustration. Especially since the run command isn't documented anywhere, except for https://github.com/cucumber/aruba/blob/955a84099b416576e307a69ad9831aa20718e284/features/api/command/run_simple.feature. And that file isn't easy to find by navigating from the readme.

It doesn't make sense for me to browse the whole features tree to "guess" that run_simple is under "api" and then "command". So documentation for the primary feature or Aruba ("running a command") kind of "burried" in a feature file somewhere.

Or, how am I supposed to know that examples for running something interactively are in: https://github.com/cucumber/aruba/blob/e880f1f7dfe384ef8ec7042fcce085ee17f208a5/features/steps/command/output.feature ?

It's ironic, that the very thing I want Aruba to do (running commands, interacting with them while waiting for input/output, and debugging when things go wrong) are hidden either in the source code or in "enormous" feature files that are hard to skim through and hidden inside a feature tree under names that are hard to "guess" quickly.

In short: releasing 1.x won't help me get my time back. Still better than "insufficiently documented" deprecations. If I'm going to get tons of deprecations in 1.x, I'd rather just wait for 2.x... I'm just not that interested in renaming API methods I call every few weeks. Things like that can really be put aside for 2.x, since new users are going to start with the latest version anyway.

Probably the worst change is renaming announcer tags - and then I'm hacking Aruba's codebase wondering why debugging doesn't work...

It's a bit like the only "benefit" Aruba is delivering right now are: deprecations and "better API method names".

If I could "turn back time", I'd probably just write my own solution - because I'm not really interested in Windows or "doing everything in Cucumber" or "documentation in Cucumber" or anything other than Ruby-2.x, RSpec and Linux, really. I'm also unlikely to work on the same projects for 10 years, so if releases take months, I'll likely be gone by then. (I've already given up on using Aruba for a few projects already).

I'd rather have a crappy (or bare-bones) 1.x with "quick-PR-review-and-release" cycles to make up for it.

I really wish I knew earlier that Aruba is a bad match for me. I just want to "talk" with an app interactively, control it's environment and have test fail with understandable errors if something goes wrong. Ideally something I can easily copy and paste and use as a new project, without any setup.

So it's not about "1.x" or "Semver" as much as: having an estimation of my costs vs my benefits. (And that goes both towards using Aruba as well as contributing).

ghost commented 8 years ago

@e2 I tried to compile a list of this we can work based on from your comment.

Having tons of deprecations in a 0.x version is really weird and confusing.

Removal of deprecated code is now tracked here #396 .

Especially since the run command isn't documented anywhere, except for https://github.com/cucumber/aruba/blob/955a84099b416576e307a69ad9831aa20718e284/features/api/command/run_simple.feature.

What abou this? https://github.com/cucumber/aruba/blob/master/features/api/command/run.feature

And that file isn't easy to find by navigating from the readme.

That's tracked by #269 and unfortunately a known issue.

It doesn't make sense for me to browse the whole features tree to "guess" that run_simple is under "api" and then "command". So documentation for the primary feature >or Aruba ("running a command") kind of "burried" in a feature file somewhere.

Have you seen https://github.com/cucumber/aruba#user-documentation? Would it help to have some documentation from the feature files rendered to HTML which is easier found by "Google" and the like? Maybe something similar to http://www.relishapp.com/cucumber/aruba/docs? Which was the first try to get documentation which is easier to access. Unfortunately it didn't work that well as you can see in the output.

Probably the worst change is renaming announcer tags - and then I'm hacking Aruba's codebase wondering why debugging doesn't work...

That should have not happened. Sorry for that. You mean one of those? https://github.com/cucumber/aruba/blob/master/lib/aruba/cucumber/hooks.rb#L63

If I update Aruba on an older project I get lots of deprecations. History.md doesn't really help me get through them. If a method is renamed, that's ok. If there' something bigger changed, I want to know how to do "X", especially if there's a new way to do that.

This is now tracked by #397

e2 commented 8 years ago

Have you seen https://github.com/cucumber/aruba#user-documentation?

If I'm a Cucumber user, I only have ONE question: "what steps does Aruba provide?".

Now, image Google search, except that without an input field you have all the possible categories of answers - and you had to find the answer you want by navigating the tree. Seems like a good idea from Google's perspective, but not too useful for users, right?

Now, sure, there's this: http://www.relishapp.com/cucumber/aruba/v/0-11-0/docs/steps/overview-of-steps

But, what I want is the RESULT of that command. It doesn't make sense for me to clone aruba and copy and paste that grep expression every time I need a reference of all the available steps.

There are only 82 steps. 28 "Given", 22 "When" and 32 "Then". That's only 3 categories and about 20 entries each. That's not overwhelming for a cheatsheet. It's also a lot less intimidating than History.md.

Meanwhile, I don't need 10 example scenarios of how to use write_file. To some who has ever needed File.write or IO.write or Pathname#write, there's little confusion in how to use or how write_file works.

The biggest flaw of Cucumber was: expecting business people to use it to create steps. I've seen project managers invent their own "version" of TDD, where one developers was supposed to write tests for another. At the end of the day, even software developers rarely get the steps right the first time, so there's rework. It's quite hard to create reusable steps that are just the right "size" and will survive unmodified across projects.

And because steps are so prone to "updates", it's why memorizing them makes no sense. Which means an up-to-date "cheatsheet" is essential. With Aruba's current documentation, I have to pretty much extract such a cheet sheet on my own. And every release a few steps may be deprecated, so now in reality I'm a "documentation maintainer" if I want to be one or not.

What abou this? https://github.com/cucumber/aruba/blob/master/features/api/command/run.feature

This would be a much better README example than things like write_file. Think of a copy-and-paste for a typical use case, which includes:

  1. Running a cli app with parameters
  2. Testing the stderr, stdout and exit code
  3. Running a cli app interactively with multiple timeouts for reading and expecting input (e.g. at least "prompts". If you're out of practical ideas, consider an IRB or Pry session where a command can take a few seconds (5-10) and potentially fail with an error during one command.
  4. With announcer on by default, so if the output doesn't match, at least the user can see the error. (NOTE: often if people are capturing STDOUT, the example should first check that STDERR is empty, and then that the exit code is 0, and only then check the STDOUT).

Currently, Aruba seems to "favor" only succeeding scenarios. So it's favoring documentation (the easy part) over development (the hard part to get right). The problem is: no matter how beautiful the documentation is for successful scenarios, if you discourage developers by "hiding" useful tools and functions by default ... it's game over anyway, because they aren't going to want to use Aruba, except for documenting what already works.

Unfortunately it didn't work that well as you can see in the output.

I think Aruba has an identity crisis. It wants to present itself as a BDD tool, even though it's designed and prioritized as a "documentation tool". On top of that it technically has an API, although it's more like a DSL, and so Aruba is "embarassed" and thus tries to "hide" it's API.

This means Aruba actually has 2 "interfaces": Cucumber steps and the "DSL". Take RSpec as an example - I have no idea what the internal classes are, and they're never mentioned in the docs. And I don't need them. E.g. I never need to explicity use RSpec::Core::Example, I only use it as:

 config.around do |example|
    foo.bar()
    example.run
    foo.baz()
  end

and that's exactly how it's documented: http://www.rubydoc.info/gems/rspec-core/RSpec/Core/Example

It's called a "DSL" and not an "API". That's the distinction: around do |example| is the DSL, and RSpec::Core::Example is the API.

So the term Aruba API suggests it's for advanced developers trying to create Aruba extensions.

You mean one of those?

Yes. Though the term "hooks" is misleading. They suggest there's something in the Aruba API that I can "hook into" for callbacks, etc. If you don't call them hooks in Cucumber, it doesn't make sense to call them something else when used for RSpec.

Again, I don't have a RSpec cheatsheet for Aruba, so the best place is to look at Aruba's Cucumber steps. And there is no cheat sheet for those either. (Ideally, I'd just want to find the Cucumber steps I need, and then transplant them into my RSpec code - usually that's possible if the documentation has "source" links for steps and methods).

Overall, I don't want to learn Aruba or dabble in it's internals just to fix deprecation warnings.

I really expect to be able to quickly use Aruba just like I do with https://github.com/rspec/rspec-rails . Even if I don't use RSpec::Rails for ages, it's easy to find out how to use it with Rails 4 and RSpec 3, whether I'm writing routing specs or view specs or whatever. And that's all just from the README.md.

Aruba is also confusing in where the project is at:

  1. rspec-rails was 2.0.0 since 2009, currently it's 3.5.0.beta2. Aruba is still 0.x, which suggests the "api can change at any time", which suggests people should not rely on the API. Developers thus will be quite forgiving for API changes, as long as upgrade notes are decent. According to SemVer, it's reasonable to lock to minor versions anyway, so breaking compatibility form 0.14.x to 0.15.x really isn't a big deal - because at 0.x developers are expected to lock to minor versions, so they only get critical bugfixes if they don't have time to upgrades.
  2. Yet, Aruba "insists" on heavy use of deprecations for an 0.x version, which suggests stability and backward compatibility is more important than anything else. By analogy, the "policy against changes" matches probably what RSpec 2.99 is for (compatible with 2.x by allows gradual migration through options towards 3.x).
  3. Version 0.x suggests there's a lot of activity going on to define the API. This suggests are PRs for new features have to be rejected until 1.x, so contributors are "expected" to wait until 1.x.
  4. But, 1.x. hasn't been released for about a year, which shows that the project had ambitions to be useful, but ended up dead before it became useful (a 1.x. would suggest).
  5. Then again, attempts to drop features and support to create a 1.x as a "bare-bones" base for new features was met with heavy scrutiny as if 0.x was production-level with only "security patches" expected.
  6. Not only that, but bug fixes are not getting priority releases. (In Agile, fixing bugs and releasing those fixes is the highest-priority activity, which exception only being security vulnerabilities). This suggests the project is either extremely busy with "soon-to-be-delivered" changes, or all the maintainers died from the Black Plague or something.
  7. If the project didn't get a 1.x over a year, that suggests the maintainers don't have time. Then, again, making contribution a grind doesn't seem like a good strategy to get more help. Contribution and sharing should be less costly in the long run than permanently maintaining custom forks.
  8. I complain and I fix what I complain about. When there's nothing left to complain about, I improve things. In this project, the first seems to conflict with a CoC (I won't get into that, but it is/was a fact). The latter seems to conflict with "backward-compatibility". Even for a 0.x version.
  9. There's heavy focus on documentation, but there's also heavy focus on keeping the codebase as it is. It seems that Aruba releases are mostly just "documentation releases" and not "feature releases". So it's like the sole purpose of releasing Aruba 1.x is to release "documentation updates for Aruba". It's like the methods are renamed solely to "look better in the docs". That's an overstatement sure, but it's also the opposite of what I want to focus on: saving time during development instead of achieving "nice documentation that no one will read". And my goals don't have to be aligned, sure - but that doesn't mean the codebase has to "wait" for documentation updates. Maybe it's better to have a separate project call aruba-docs or aruba-tutorial? Because the idea should be always to create a core for professional use, and build "user friendliness" around the core, and not the other way around. (As Aruba tries to do).
  10. I don't think there's a lot of sense in promoting things like javac hello.java, unless you're providing actual interpreter-specific features. It's much better to just point to a real-life Java project using Aruba, so people just investigate the codebase there. aruba-java would be better, because you could spin up the VM on startup, add/remove Java debugging options, have an API for limiting memory, etc. Otherwise, Aruba can only be the "lowest-common denominator". Not to mention: people should be encouraged to use custom steps anyway, e.g. "Given I run my hello app from a jar", which could reuse the "run" steps with the right parameters, etc.
  11. Also, people generally aren't programming language "poliglots", unless that's a specific feature of Aruba to be able to test bindings of a library for multiple platforms, by using example apps in different languages. E.g. what I really want is aruba-ruby-rspec-linux and without the Cucumber part. So in-process becomes useful (while it wouldn't be as useful for a Java developer).
  12. The way things are I'd at least need a "glossary" to know what the difference is between API and DSL (there is none here) and between "announcer" and "hooks" (there also is non here) and between "command" and "API method" (there also is none here) and between "process" and "last command" (there also is none here, except that "command" also means Aruba API method) and between "Aruba documentation" and "Aruba feature files used for testing" (there is not difference here either) and between "debugging" and "announcing" (there is a difference here) and between "cucumber hooks" and "Aruba hooks" (big difference here!).

Overall, I'm biased in that I'm no longer an "Aruba beginner". What I remember is being utterly confused when what was supposed to be easy wasn't. I kept on finding out that I wanted to do something "obvious" and "typical" with Aruba, and I always ended up having to dive into the code to find answers quickly. For example: how do I wait for an interactive process to stop after typing in 'exit'? will stop kill it or will it wait? Do I really have to "run it and see" to find out? What does run return? Can I call stop on the object returned?

Also, documentation navigation is a barrier. The most discouraging is to make 3 clicks just to get to a folder with only one file, then to go back up and "try another folder". So a cheatsheet would help a lot. (One that would be guaranteed not to be deprecated and won't require "generating" a list yourself).