launchdarkly / ruby-eventsource

Server-Sent Events client for Ruby
Other
40 stars 16 forks source link

ci: Add SSE contract test support #25

Closed eli-darkly closed 9 months ago

eli-darkly commented 3 years ago

See: https://github.com/launchdarkly/sse-contract-tests

An example of this running in CI is here. That workflow is currently failing due to problems that appear to be JRuby-specific (see below); in all of the non-JRuby jobs, everything passes and I believe the SSE implementation is now fully compliant with spec.

The two kinds of failures I'm seeing in JRuby (which I haven't yet filed separate stories for, because they're so odd that I need to rule out problems in the test logic first) appear to be problems in HTTP client behavior, not in SSE parsing:

If we can fix those, then we will have much better test coverage in JRuby than before, since as described in https://github.com/launchdarkly/ruby-eventsource/pull/27, the mechanism we've been using to do end-to-end HTTP tests within the unit tests for this project was extremely unreliable in JRuby.

If you'd like to test this locally in any version of Ruby, clone this repo and also https://github.com/launchdarkly/sse-contract-tests. In ruby-eventsource, run make build-contract-tests and then make start-contract-test-service. Then, in sse-contract-tests, run make and then ./sse-contract-tests -url http://localhost:8000 - that's it.

keelerm84 commented 11 months ago

I have updated this PR so it runs on GH actions instead of circleci.

The problems Eli mentioned still seem to be present, so I've just disabled the contract tests in jruby for now. Might as well get some value from them now instead of waiting until the jruby stuff gets resolved (if it ever does).