GoToDefinitionTests and maintenance pain

[smoothdeveloper]

I'm adding a small change to the signature file generator.

That change gets 50 tests to fail (which is fine) although as it is there is no clear way for me to easily fix the expected results in each failing tests.

Moreover, over the 57 tests in this class, the test implementation is basically repeated over and over, with input file and expectation being inline as raw strings in the code.

I don't think this structure is maintainable / scalable to encourage adding more tests or enhancing the feature (generally involves breaking all those tests with need to adjust expected output), so I would propose the following:

• make a tests/data/gotodefinition folder, with a subfolder per test we currently have
• migrate our test methods to each subfolder: one input file, one json file (for giving the position and eventual additional arguments), one expected output file
• replace our 57 tests with a single iteration over the folders (adding a test involves 0 code) taking care of accurate and detailed error reporting
• consider adding facilities to overwrite expected output file with new results (to be reviewed in a normal git status diff, this is useful when we change signature file generation and saves us to change manually the 50 and growing current expected output)

[cloudRoutine]

I agree that the current test setup isn't ideal, but it'd be nice to see a proof of concept for what you're proposing. It'd only need to cover 2 or 3 test cases, but should show how all the different parts you're describing would work.

[dungpa]

I support the ideas.

To fix Travis unit tests, I had to go through all failing tests and fix them manually. The current process is extremely painful.

[smoothdeveloper]

Could someone run the test I've submitted in my PR (#1452) / branch: https://github.com/smoothdeveloper/VisualFSharpPowerTools/tree/gotodef-test-refactor to confirm it runs locally?

I need to figure out why they fail on appveyor/travis but want to confirm it runs locally and might be relative to wrong version of FSharp.Core used for signature generation in the tests.

[smoothdeveloper]

1452 is ready for review