search

flowbased / fbp-protocol

Tests, schemas, and specifications for the Flow Based Programming Network Protocol
https://flowbased.github.io/fbp-protocol/
MIT License
40 stars 7 forks source link

Add a test using fbp-spec #5

Closed jonnor closed 9 years ago

jonnor commented 9 years ago

fbp-spec exercises the FBP protocol implementation of a runtime quite a lot, and is generally useful, so we could include it here.

A testcase for core/Repeat (like https://github.com/flowbased/fbp-spec/blob/master/examples/simple-passing.yaml) but covering all JSON types would be a really good start.

It can be trivally integrated with Mocha like this https://github.com/flowbased/fbp-spec/blob/master/spec/mocha.coffee

chadrik commented 9 years ago

I've been wondering if it wouldn't be better to merge these two repos. It's unclear from the docs what the difference between them is. What's the advantage of keeping them separated?

On Sun, Aug 9, 2015 at 9:26 AM Jon Nordby notifications@github.com wrote:

fbp-spec exercises the FBP protocol implementation of a runtime quite a lot, and is generally useful, so we could include it here.

A testcase for core/Repeat (like

https://github.com/flowbased/fbp-spec/blob/master/examples/simple-passing.yaml ) but covering all JSON types would be a really good start.

It can be trivally integrated with Mocha like this https://github.com/flowbased/fbp-spec/blob/master/spec/mocha.coffee

— Reply to this email directly or view it on GitHub https://github.com/flowbased/fbp-protocol/issues/5.

jonnor commented 9 years ago

They serve different purposes:

It is not the purpose of fbp-spect to test the FBP runtime protocol implementation. But because it uses the protocol heavily it adds some coverage as a side-effect.

There is also no shared code between the two projects, so I don't see any reason to merge them...

jonnor commented 9 years ago

@chadrik did my explanation above make sense?

chadrik commented 9 years ago

I see how those are two different use-cases, but I disagree that they are so different as to require separate repos. If testing my components requires that the FBP protocol is properly implemented by my runtime, then to me it makes sense that testing the protocol be part of my tests. Otherwise errors might occur that appear to be my component's fault when in fact they are at the protocol level. And if this ticket is an indicator that we're considering adding a test from what was supposed to be the higher-level repo into the lower-level one, than I think that circularity confirms that these two repos should be one.

As an outsider getting started with noflo I found the number of inter-related and similarly named repos in the noflo project bewildering, and I still do. I don't think we should repeat that here. What is the goal of the repo-level division here? To simplify the life of the runtime author by not burdening them with component tests? To me it just seems more complex. I think we should have just one repo for testing everything together at once, which also has the ability to run particular test cases, as any test runner would. If that means folding fbp-protocol into fbp-spec, then great. We should then update the readme to give a concise high-level explanation of the test suites available and who should use them for what.

my 2 c.

jonnor commented 9 years ago

A goal is that application developers don't need to install and run things which the developer of runtime uses to ensure the runtime/protocol implementation is correct. One cannot expect app devs to run

fbp-spec is also a fairly unproved testing concept (declarative, data-driven, multi-runtime). We've already had several testing tools/projects in NoFlo which have more-or-less failed. I don't want to pollute fbp-protocol code with such experimental things (calling fbp-spec as a command-line program would be a very loose coupling compared to merging codebases).

jonnor commented 9 years ago

Anyway, we can close this issue as a wont-fix. Runtime developers who find fbp-spec to be a useful tool in verifying their FBP protocol implementation can then use it directly instead of through fbp-protocol (or not).