Tracking Testing Issues

BenderScript commented 5 years ago

[x] go test ./... does not succeed or at least after waiting 10 minutes there was no output.

I understand make test performs some prep work for integration test but an often cited Go golden rule of testing is that go test ./... should run unit tests unencumbered right off the bat.

[ ] Tests create files in the root directory. We should move them to a test output directory.
[x] make test itself fails, need to dig deeper as to why this is the case.

remingtonc commented 5 years ago

I think #8 will resolve the 3rd point. Try running make test with it.

remingtonc commented 5 years ago

@repenno Do you have any ideas on how to make the tests create files in the test output directory? I have been reading go docs and it looks like we have to test from the package base. Do we need to modify the code to output tests to a test output directory?

BenderScript commented 5 years ago

Yes, we need to change the code to output to the right (a) directory. But there are a couple of issues I've been investigating:

unit tests uses func startup() that sets the Logfile name and indirectly location since we are not using relative paths. This is an issue in itself
The binary ((or main()) also uses func startup() which means the Logile name and location could potentially be the same as for testing causing overwrite across purposes and runs. This is problematic.

In order to keep things small and take baby steps I think we first should use separate Logfile names and locations for main() vs. test. We can then look into the relative path issue when we work on the project structure.

Thoughts?

remingtonc commented 5 years ago

Sounds smart to me. Does #18 address your initial test issues, and we can address the file output locations in another issue?

BenderScript commented 5 years ago

@remingtonc Is your intention that make test passes or just aim to create different targets? make test does not work for me. Check if you can reproduce it but please remove all Kafka and Zookeeper containers and images in case you haven't.

BTW, you probably also noticed the tests spill out hundreds of lines under ===A=== and ===B===. I have no idea what these are but certainly add to the confusion. IMHO tests should output name of test and pass/fail. Will open a separate issue for these.

=== RUN TestUDPServer/BurstDenseGPB --- PASS: TestUDPServer (11.52s) --- PASS: TestUDPServer/BurstSparseGPB (6.50s) xport_udp_test.go:228: Complete: handled 10000, sent 10000, max 10000 --- PASS: TestUDPServer/BurstMediumGPB (2.50s) xport_udp_test.go:228: Complete: handled 10043, sent 10043, max 10000 --- PASS: TestUDPServer/BurstDenseGPB (2.51s) xport_udp_test.go:228: Complete: handled 10099, sent 10487, max 10000 FAIL exit status 1 FAIL github.com/cisco-ie/pipeline-gnmi 48.678s make: *** [test] Error 1

remingtonc commented 5 years ago

@repenno If the tests are completing now via branch test-fix, and Kafka is not included, that is a success for this specific issue. make test (in test-fix) should only execute unit tests that do not depend on any other infrastructure. make testall will spin up Kafka/Zookeeper and execute the Kafka tests as well. The tests failing (and involving the hundreds of lines of output) is due to #11 where there is a very small difference and I cannot determine which should be the correct. The test result might be more correct than the current test data, if that makes sense.

BenderScript commented 5 years ago

As we discussed, we are going with this fix

cisco-ie / pipeline-gnmi

Tracking Testing Issues #10