search

open-feature / ofep

A focal point for OpenFeature research, proposals and requests for comments
https://openfeature.dev
20 stars 15 forks source link

POC for cross-platform test cases #11

Closed toddbaert closed 2 years ago

toddbaert commented 2 years ago

With so many language to support, it would be nice to have a simple cross-platform testing framework for basic SDK behavior. It could be a set of inputs/states/outputs defined in JSON, which a test harness implemented in each language would parse and use to evaluate the function of the respective SDK. Harness has done something like this for their numerous SDKs: https://github.com/drone/ff-test-cases, https://github.com/harness/ff-nodejs-server-sdk/blob/main/__tests__/evaluator.test.ts cc @davejohnston

I'd like to propose a POC in 2 languages. I will volunteer for the JS implementation. It would be great if we could get one other as well.

davejohnston commented 2 years ago

So this has worked well for us. I would be interested in using cucumber.
The upside is that there is good support across a bunch of languages, so generating a boiler plate test from the test description would be nice. The down side is for lanagauges which don't have cucumber driver, it might be more complex to parse the test cases than just using JSON.

Notionally we could define a test case like:

Feature: Test Flag returns true when on

Given: A Boolean Flag
And the state is on
And the default on variation is true
And the default off variation is false
When I evaluate the flag
Then I get on

Again a nice feature is we can use tables here to test alot of this.

toddbaert commented 2 years ago

I'm working on doing a PoC with cucumber/gerkin.

toddbaert commented 2 years ago

Closed with: https://github.com/open-feature/node-sdk/pull/70