Add Elasticsearch integration tests.

jcantrill commented 7 years ago

This PR adds a running integration test. It still will require mocking out some calls to kubernetes.

cc @portante

fusesource-ci commented 7 years ago

License check failed: run mvn -N license:format to update all licenses, commit, squash & force push please.

jcantrill commented 7 years ago

This PR superceeds #45

fusesource-ci commented 7 years ago

Tests failed.

fusesource-ci commented 7 years ago

Tests failed.

fusesource-ci commented 7 years ago

Tests failed.

fusesource-ci commented 7 years ago

License check failed: run mvn -N license:format to update all licenses, commit, squash & force push please.

fusesource-ci commented 7 years ago

Tests failed.

fusesource-ci commented 7 years ago

License check failed: run mvn -N license:format to update all licenses, commit, squash & force push please.

fusesource-ci commented 7 years ago

Tests failed.

fusesource-ci commented 7 years ago

License check failed: run mvn -N license:format to update all licenses, commit, squash & force push please.

jcantrill commented 7 years ago

Discovered our plugin is not executing. Looks like its because the boot sequence is not calling our plugin and its dependent services

jcantrill commented 7 years ago

Found the issue, it was in my test. The fix exposes the need to mock the api server

fusesource-ci commented 7 years ago

License check failed: run mvn -N license:format to update all licenses, commit, squash & force push please.

jcantrill commented 7 years ago

@portante getting this error. Not sure what is happening here. Seems like some sort of internal communication I need to configure.

2017-10-09 16:03:10 INFO  allocation:238 - [openshift_test_1] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[.kibana.5a7b59d58c20ec4352c60cb529532932fdad1990][0]] ...]).
2017-10-09 16:03:10 ERROR DynamicACLFilter:125 - Error handling request in DynamicACLFilter
MapperParsingException[failed to parse, document is empty]
    at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:153)
    at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:309)
    at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:584)
    at org.elasticsearch.index.shard.IndexShard.prepareIndexOnPrimary(IndexShard.java:563)
    at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnPrimary(TransportIndexAction.java:211)
    at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:223)
    at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:157)
    at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:66)
    at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:657)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:287)
    at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
    at com.floragunn.searchguard.ssl.transport.SearchGuardSSLTransportService.messageReceivedDecorate(SearchGuardSSLTransportService.java:172)
    at com.floragunn.searchguard.transport.SearchGuardTransportService.messageReceivedDecorate(SearchGuardTransportService.java:191)
    at com.floragunn.searchguard.ssl.transport.SearchGuardSSLTransportService$Interceptor.messageReceived(SearchGuardSSLTransportService.java:113)
    at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:77)
    at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:378)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

fusesource-ci commented 7 years ago

Tests failed.

jcantrill commented 7 years ago

@lukas-vlcek @portante done!

fusesource-ci commented 7 years ago

Tests failed.

lukas-vlcek commented 7 years ago

@jcantrill I am looking at this now:

Seems you did not use Integration test support from Elasticsearch so we need pom.xml cleanup (bunch of things should be taken out)
I noticed some files (like the pom.xml) has messed up licence header
Do we need .travis.yml file? It does not seem we are using Travis for tests, are we?

I can follow up with clean up PR if you want. Let me know.

Apart from this LGTM. Are we able to run ITs on ES cluster with more than a single node?

jcantrill commented 7 years ago

@lukas-vlcek I got anxious to merge! Regarding your questions:

Seems you did not use Integration test I would have prefered to use theirs but there were start hooks they had which they did not allow overrides so you could not do any customization.

support from Elasticsearch so we need pom.xml cleanup (bunch of things should be taken out)

Can you do that or point me to what to remove? I have a BZ to resolve that I am going to validate using this code

I noticed some files (like the pom.xml) has messed up licence header

I saw this as well. It will be fixed in forthcoming PR

Do we need .travis.yml file? It does not seem we are using Travis for tests, are we?

We do not use Travis but fabric8 CI. Do we have reason to hook up travis? If not then we should remove it.

I can follow up with clean up PR if you want. Let me know.
Apart from this LGTM. Are we able to run ITs on ES cluster with more than a single node?
You should be able to. I yanked this code from SearchGuard and if you look at the start method it allows to boot ES multicluster. There may need to be some refactoring to properly support our use case but it is possibl

lukas-vlcek commented 7 years ago

@jcantrill

I will prepare cleanup PR to remove all unnecessary deps from pom.xml
I originally added the Travis support but we do not need it now (using other CI), I will remove it too
Do you want me to remove messed up license header if I find any or this will be taken care by other PR?

lukas-vlcek commented 7 years ago

Cleanup RP https://github.com/fabric8io/openshift-elasticsearch-plugin/pull/107

lukas-vlcek commented 7 years ago

Just for the record I am adding reference to another option about ES integration tests. It might be useful in the future or in different use cases. There is elasticsearch-maven-plugin but ATM it might not be fully ready for security aspects integrations, see #42.

fabric8io / openshift-elasticsearch-plugin

Add Elasticsearch integration tests. #105