Discussion on jasmine test organization and abstractions

[etpinard]

@mdtusz @mdtusz

Our jasmine test suites are grown a lot in the past few months. We should spend some time discussing ways to make writing jasmine tests easier.

[mdtusz]

I find the biggest drain in writing tests to be the boilerplate and ensuring that there's no leaks or side affects from test to test - often I'll find myself debugging tests that pass individually but not when run together. I'm not sure the best way to go about improving this, or if we really can do much other than improving upon the "standard" way of setting up/tearing down each test.

Perhaps if we take some time to look through existing tests and seek out the most commonly recurring patterns, and create a small stdTestLib including the things we always use (create/destroy the gd followed by Plotly.plot(...).then(...), etc.)?

[monfera]

Here's an attempt that we decided not to use for now, and it would benefit from lots of constructive criticism even if we were to use test utilities that are heavier than createGraphDiv etc.:

Use: gl_plot_interact_basic_test

Backstage:

• with_setup_teardown
  • fail_test

Notes, so as we remember considerations that went into it:

• it uses withSetupTeardown, which is a self-contained if a bit heavy abstraction
• no use of global gd or other global; no use of beforeEach or afterEach, inspired by point 3 of this document: no state sharing among tests
• contains (obviously) the setup and teardown including the Purge as advised by @etpinard
• also protects against silent failure: with promises, if there's an error inside a .then function, it'll fail silently, causing the belief that the tests passed, even though the tests past the failure point hadn't run
• protects against accidentally not calling done; in fact, one needs not to call done even with async
• however if test makers call done themselves (for some reason, e.g. at the end of the deepest setTimeout if the test is not fully promise based), then it'll still perform the silent failure catch as well as the teardown
• facilitates easier split / merge / reorg of test files, because test cases no longer depend on globals or beforeEach/afterEach
• the test file just requires a single with_setup_teardown rather than createGraphDiv, destroyGraphDiv and perhaps the new failTest
• with a single commenting out of the teardrown call in with_setup_teardown, a jasmine test can be coerced to retain the plot, which is useful with fdescribe or fit during debugging or TDD iterations.

The docs in the source code overlap with the above. Also, as it's a first iteration, there may be shortcomings relative to the above aims, despite having tested tests, and there may be technical weaknesses besides the issue that it's a much heavier abstraction than createGraphDiv.

[etpinard]

Just noticed this ticket again.

This ticket never turned into a serious discussion that could possible lead to changes in our test suites.

Closing.