Add more container-based 3rd party application tests

[smlambert]

As an example of this type of testing, we have added scala tests: https://github.com/AdoptOpenJDK/openjdk-tests/tree/master/thirdparty_containers/scala

We have derby and solr_lucene tests running in the same "dockerized" way, so we should also add those test sub-folders.

Once we have those 3 examples of 3rd party application tests added, we should consider the list of other applications we would like to add and prioritize them (eclipse, cassandra, etc). It will also be good to engage with these communities so that they are aware that we will be running their community test suites against the different flavours of JVM at AdoptOpenJDK (openjdk8, openjdk9, openjdk+openj9...).

Here is a list of potential ones to add, not in any priority order:

• [x] scala - https://www.scala-lang.org/ - added
• [x] openliberty - https://github.com/OpenLiberty/open-liberty - in progress #367
• [ ] openwhisk - https://openwhisk.apache.org/
• [x] derby - https://db.apache.org/derby/
• [x] solr/lucene - http://lucene.apache.org/solr/
• [x] wildfly - http://wildfly.org/
• [x] payara microprofile tck - https://www.payara.fish/
• [x] thorntail (wildfly-swarm) microprofile tck - http://wildfly-swarm.io/
• [x] open-liberty microprofile tck - https://github.com/OpenLiberty/open-liberty
• [x] apache tomee http://tomee.apache.org/
• [x] tomcat - http://tomcat.apache.org/
• [x] jenkins - https://jenkins.io
• [x] elasticsearch - https://www.elastic.co/products/elasticsearch
• [ ] bazel - https://bazel.build/
• [ ] zookeeper - https://zookeeper.apache.org/
• [ ] akka - https://akka.io/ - in progress #422
• [ ] cassandra - http://cassandra.apache.org/
• [ ] flume - https://flume.apache.org/
• [ ] activemq - http://activemq.apache.org/
• [ ] storm - http://storm.apache.org/
• [ ] alfresco - https://www.alfresco.com/products/community/download
• [ ] geode - http://geode.apache.org/ - in progress #543
• [ ] hbase - https://hbase.apache.org/ - in progress #544
• [ ] hibernate - http://hibernate.org/
• [x] kafka - https://kafka.apache.org/
• [ ] eclipse IDE - https://www.eclipse.org/ - this is non-trivial if we want to pick up nightly builds of eclipse, which are not currently served up in an easy to consume way. we can/should decide to take last release for this testing to simplify... related: https://github.com/AdoptOpenJDK/openjdk-tests/issues/1000
• [x] quarkus - https://github.com/quarkusio/quarkus.git
• [x] quarkus-quickstarts - https://github.com/quarkusio/quarkus-quickstarts.git
• [x] camel - https://github.com/apache/camel-quarkus.git
• [x] wycheproof - https://github.com/google/wycheproof.git
• [ ] logstash - see https://github.com/AdoptOpenJDK/openjdk-tests/issues/243
• [x] jacoco - https://github.com/jacoco/jacoco - re: https://github.com/AdoptOpenJDK/openjdk-tests/issues/1697
• [x] netty - https://github.com/netty/netty
• [x] spring - https://github.com/spring-projects/spring-boot

Also, this Testing Java link and this Quality Outreach link relates to this effort (engaging with Java projects, unblocking them if they have issues)...

Consider new tests via external_custom test target.
Consider startup tests for quick smoke test of each application.

[suman-mitra]

Add Elasticsearch under thirdparty container tests https://github.com/AdoptOpenJDK/openjdk-tests/issues/360

[tellison]

In additional to a broad set of application tests to prove the quality of the binaries, it would be useful to have a suite of MicroProfile tests to demonstrate that our binaries are a good foundation for MicroProfile.

Microprofile doesn't have a reference implementation, rather there are a number of popular implementations that cover different proportions of the specification.

The following projects will provide a good test environment for our binaries:

• OpenLiberty,
• WildFly Swarm,
• Payara, and
• Apache TomEE.

The tests would be taken from the MicroProfile TCKs, e.g. fault tolerance APIs, metrics APIs, etc. Note that not all the implementations implement every part of the MicroProfile spec, so we'd need to define the appropriate set of TCKs to run against each implementation.

As the MicroProfile projects are open communities, I expect we can get help in putting together the tests and running them on our test farm.

[kwsutter]

This is a little bit longer term, but you should also consider the use of the Jakarta EE test suite (CTS) when that becomes available under Eclipse licensing. I know from personal experience that running the CTS can be complicated and time-consuming. So, maybe we can work with the various vendors and developers that have or will be running the Jakarta EE CTS to also run with AdoptOpenJDK binaries. Not as a requirement, but as a "friend of open source". We're still a few months away before these test suites will be available and usable, but something to keep in your back pocket for future testing efforts.

[smlambert]

Thanks @tellison and @kwsutter - we are in process of updating our test doc to state our goals around running these external tests and engaging with the open-source communities (Summarized mission: us running their tests, them using adopt SDKs, attempting to make both aspects of testing drop-dead easy).

Both your suggestions fit in with the overall mission. 👍 Link to updated mission statement to come shortly.

[smlambert]

related: https://github.com/AdoptOpenJDK/openjdk-support/issues/30

[joeyleeeeeee97]

Does mybatis worth adding if we are considering hibernate?

[llxia]

We may also consider adding jython test suite: https://github.com/jython/jython/tree/master/tests Related openj9 issue: https://github.com/eclipse-openj9/openj9/issues/12238

[joeyleeeeeee97]

Maybe consider adding rocketmq https://github.com/apache/rocketmq