Kafka integration test can fail during setup

elastic / logstash

Logstash - transport and process your logs, events, or other data

https://www.elastic.co/products/logstash

Other

75 stars 3.5k forks source link

Kafka integration test can fail during setup #7178

Closed andrewvc closed 7 years ago

andrewvc commented 7 years ago

Form https://travis-ci.org/elastic/logstash/jobs/234760314

  1) Test Kafka Input can ingest 37 apache log lines from Kafka broker
     Failure/Error: @fixture = Fixture.new(__FILE__)

     Docker::Error::UnexpectedResponseError:
     # /home/travis/.rvm/gems/jruby-1.7.25/gems/docker-api-1.33.4/lib/docker/util.rb:200:in `extract_id'
     # /home/travis/.rvm/gems/jruby-1.7.25/gems/docker-api-1.33.4/lib/docker/image.rb:265:in `build_from_tar'
     # /home/travis/.rvm/gems/jruby-1.7.25/gems/docker-api-1.33.4/lib/docker/image.rb:278:in `build_from_dir'
     # ./services/kafka_service.rb:10:in `setup'
     # ./framework/fixture.rb:73:in `setup_services'
     # ./framework/fixture.rb:72:in `setup_services'
     # ./framework/fixture.rb:29:in `initialize'
     # ./specs/kafka_input_spec.rb:12:in `(root)'

jsvd commented 7 years ago

this seems to be a dup of an issue that got a PR and was closed, I added a comment there: https://github.com/elastic/logstash/issues/7160#issuecomment-303061906

andrewvc commented 7 years ago

@jsvd would you like to use this as the new issue or close / reopen #7160 ?

jsvd commented 7 years ago

I'll let @original-brownbear decide if this is similar (then close) or a different thing (then keep open).

andrewvc commented 7 years ago

I believe https://github.com/elastic/logstash/pull/7179 will be a good fix for this, waiting for those tests to pass ATM

jsvd commented 7 years ago

I got this again in https://travis-ci.org/elastic/logstash/jobs/237106207#L1837

original-brownbear commented 7 years ago

@jsvd wasn't there some SSL bug in Jruby ? :D The error so looks like some SSL lib issue (admittedly could be a bunch of other things too), I've never seen this one with MRI Ruby ...

original-brownbear commented 7 years ago

nevermind ... Travis shouldn't be running an encrypted Docker socket ...

colinsurprenant commented 7 years ago

removing elasticsearch-bot @andrewvc can you please reassign?

jsvd commented 7 years ago

a few more instances of this: https://travis-ci.org/elastic/logstash/jobs/237429275#L2256 https://travis-ci.org/elastic/logstash/jobs/237429282#L2217 https://travis-ci.org/elastic/logstash/jobs/237429280#L2192

original-brownbear commented 7 years ago

@jsvd @andrewvc this is getting kind of painful I guess ... unless someone else has the time should I take a look here tonight?

jsvd commented 7 years ago

I can take a stab at it thought I've been avoiding to since my docker knowledge is slim. anyway it can definitely wait until you're back home. I've just been registering here the occurences so we know the frequency, and it's a good thing that it is so frequent (as opposed to once in a while).

original-brownbear commented 7 years ago

We can try a hack like https://github.com/elastic/logstash/pull/7264, let's see if we can make Travis run from a more recent Docker version. I can't reproduce this issue at all locally, but there are a number of bugs in that Docker version that may cause this ... maybe we get a freebie :P

original-brownbear commented 7 years ago

@andrewvc @jsvd haver a look at #7264 ;)

original-brownbear commented 7 years ago

fixed via #7264

original-brownbear commented 7 years ago

happens still, see https://travis-ci.org/elastic/logstash/jobs/238786856#L1900 (his the version upgrade in it)

original-brownbear commented 7 years ago

another case https://travis-ci.org/elastic/logstash/jobs/240206997#L1831

original-brownbear commented 7 years ago

link https://github.com/swipely/docker-api/issues/455 -> https://github.com/moby/moby/issues/15759

original-brownbear commented 7 years ago

Fixed at last by #7178 :)