piotrmurach / tty

Toolkit for developing sleek command line apps.
https://ttytoolkit.org
MIT License
2.5k stars 78 forks source link

e2e testing doesn't feel good #59

Open matti opened 4 years ago

matti commented 4 years ago

adding method_option :services, type: :boolean

causes tests (correctly) to fail:

  1) `test-cmd config` command executes `test-cmd help config` command successfully
     Failure/Error: expect(output).to eq(expected_output)

       expected: "Usage:\n  test-cmd config\n\nOptions:\n  -h, [--help], [--no-help]  # Display usage information\n\nCommand description...\n"
            got: "Usage:\n  test-cmd config\n\nOptions:\n  -h, [--help], [--no-help]          # Display usage information\n      [--services], [--no-services]  \n\nCommand description...\n"

and then when I copy/paste the output in

    expected_output = <<-OUT
Usage:
  test-cmd config

Options:
  -h, [--help], [--no-help]          # Display usage information
      [--services], [--no-services]

Command description...
    OUT

and I re-run the tests I get:

     Failure/Error: expect(output).to eq(expected_output)

       expected: "Usage:\n  test-cmd config\n\nOptions:\n  -h, [--help], [--no-help]          # Display usage information\n      [--services], [--no-services]\n\nCommand description...\n"
            got: "Usage:\n  test-cmd config\n\nOptions:\n  -h, [--help], [--no-help]          # Display usage information\n      [--services], [--no-services]  \n\nCommand description...\n"

And I'm giving up because this just feels stupid way to test and adjust whitespaces.

piotrmurach commented 4 years ago

Hi Matti,

Thanks for reporting this!

Well, testing strings can be a frustrating given that one character can make the whole thing fail. The rspec doesn't make it easy to compare things either, it can hide invisible characters. I'm always open to improving things, and want to know how better the end-to-end testing can be done?

As always it's not the lack of ideas but time that prevents me from improving things. Let's hope next year will bring more time and opportunities.