search

cybercongress / cyber-search

🚀 Toolchain for transactions parsing and processing
https://cybercongress.github.io/cyber-search/
Other
33 stars 14 forks source link

Kafka and Cassandra integration tests are very unstable #255

Closed hleb-albau closed 5 years ago

hleb-albau commented 6 years ago

Overview

Right now we have a lot of integration tests using embedded kafka and cassandra. The main issue with this tests, that they are very unstable. See https://circleci.com/gh/cybercongress/workflows/cyber-search

Task description

Integration tests should be stable. 100% builds for right written tests.

Required skills

Contribution rules

Here you can learn how to contribute to our project

Definition of done

Explain what you actually want to see as a result:

Extra information

Bounty amount 0.5 ETH
Experience level advanced
Project length days
Expires in 1 mounth
Bounty type bug
gitcoinbot commented 6 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

This issue now has a funding of 0.5 ETH (185.75 USD @ $371.49/ETH) attached to it.

KevinLiLu commented 5 years ago

@hleb-albau

I might be interested in taking this.

Are we constructing the embedded Kafka & Cassandra instances before and after every single test class or are the instances shared for multiple modules?

I'm been browsing the error logs in the CI and it looks like embedded Kafka often has trouble shutting down and closing.

hleb-albau commented 5 years ago

Hi @KevinLiLu,

Just start work on gitcoin to show others, that you took this task.

Right now instances are not shared, as far as I remember, but it will be nice, if they do(at least for each module)

gitcoinbot commented 5 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work has been started.

These users each claimed they can complete the work by 2 months ago. Please review their action plans below:

1) KevinLiLu has started work.

Will play around with Kafka/Cassandra configurations, and possible use 1 shared instance of Kafka & Cassandra per module.

Learn more on the Gitcoin Issue Details page.

KevinLiLu commented 5 years ago

@hleb-albau can you grant me write permissions in CircleCI so I can trigger a bunch of builds in a PR to test fail rate? The tests don't really fail on my local so it seems isolated to the CI environment.

hleb-albau commented 5 years ago

@KevinLiLu is it possible on CircleCi? I can't found required settings. I generated api token, sent to your email lu.kevin@berkeley.edu.

KevinLiLu commented 5 years ago

@hleb-albau Ah, a quick google search shows that the permissions are taken from the GitHub repo. So I would need to have write access in this repo in order to run builds...

hleb-albau commented 5 years ago

@KevinLiLu try now see

KevinLiLu commented 5 years ago

@hleb-albau got it. it works now, thanks 😄

frankchen07 commented 5 years ago

hey @KevinLiLu - Frank from Gitcoin here, are you still working on this issue?

KevinLiLu commented 5 years ago

@frankchen07 hey Frank, yes this is still ongoing.

https://github.com/cybercongress/cyber-search/pull/268

gitcoinbot commented 5 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work for 0.5 ETH (108.66 USD @ $217.33/ETH) has been submitted by:

  1. @KevinLiLu

@hleb-albau please take a look at the submitted work:

gitcoinbot commented 5 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

The funding of 0.5 ETH (107.31 USD @ $214.61/ETH) attached to this issue has been approved & issued to @KevinLiLu.