Bootstrap error in 5.4.0

[animageofmine]

Code fails to bootstrap TcpClient because it cannot register Aggregations.

Following is how I initialize it:

    val settings = Settings.builder().put("cluster.name", ElasticsearchClusterName).build()
    val esClient = TcpClient.transport(settings, ElasticsearchClientUri(s"elasticsearch://$ElasticsearchHost:$ElasticsearchPort"))

java.lang.BootstrapMethodError: java.lang.IllegalAccessError: no such constructor: org.elasticsearch.search.aggregations.metrics.percentiles.tdigest.InternalTDigestPercentiles.<init>(StreamInput)void/newInvokeSpecial
    at org.elasticsearch.search.SearchModule.registerAggregations(SearchModule.java:353)
    at org.elasticsearch.search.SearchModule.<init>(SearchModule.java:302)
    at org.elasticsearch.client.transport.TransportClient.buildTemplate(TransportClient.java:142)
    at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:268)
    at org.elasticsearch.transport.client.PreBuiltTransportClient.<init>(PreBuiltTransportClient.java:125)
    at org.elasticsearch.transport.client.PreBuiltTransportClient.<init>(PreBuiltTransportClient.java:111)
    at org.elasticsearch.transport.client.PreBuiltTransportClient.<init>(PreBuiltTransportClient.java:101)
    at com.sksamuel.elastic4s.TcpClientConstructors$class.transport(TcpClient.scala:104)
    at com.sksamuel.elastic4s.TcpClient$.transport(TcpClient.scala:112)
    at com.qualtrics.ae.api.model.ElasticsearchProfile$class.esClient(ElasticsearchProfile.scala:12)
    at com.q.api.model.ElasticsearchFieldsetAnalyticsServiceModule$$anon$1.esClient$lzycompute(ElasticsearchFieldsetAnalyticsService.scala:315)
    at com.q.api.model.ElasticsearchFieldsetAnalyticsServiceModule$$anon$1.esClient(ElasticsearchFieldsetAnalyticsService.scala:315)
    at com.q.api.model.ElasticsearchFieldsetAnalyticsServiceModule$$anon$1.esClient(ElasticsearchFieldsetAnalyticsService.scala:315)
    at com.q.api.model.ElasticsearchFieldsetAnalyticsService$class.aggregate(ElasticsearchFieldsetAnalyticsService.scala:58)
    at com.q.api.model.ElasticsearchFieldsetAnalyticsServiceModule$$anon$1.aggregate(ElasticsearchFieldsetAnalyticsService.scala:315)
    at com.q.api.model.ElasticsearchFieldsetAnalyticsService$$anonfun$aggregate$1$$anonfun$apply$1.apply(ElasticsearchFieldsetAnalyticsService.scala:45)
    at com.q.api.model.ElasticsearchFieldsetAnalyticsService$$anonfun$aggregate$1$$anonfun$apply$1.apply(ElasticsearchFieldsetAnalyticsService.scala:43)
    at scala.concurrent.Future$$anonfun$flatMap$1.apply(Future.scala:253)
    at scala.concurrent.Future$$anonfun$flatMap$1.apply(Future.scala:251)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
    at scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121)
    at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.pollAndExecAll(ForkJoinPool.java:1253)
    at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1346)
    at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.lang.IllegalAccessError: no such constructor: org.elasticsearch.search.aggregations.metrics.percentiles.tdigest.InternalTDigestPercentiles.<init>(StreamInput)void/newInvokeSpecial
    at java.lang.invoke.MethodHandleNatives.linkMethodHandleConstant(MethodHandleNatives.java:483)
    ... 26 more
Caused by: java.lang.NoClassDefFoundError: com/tdunning/math/stats/TDigest
    at java.lang.invoke.MethodHandleNatives.resolve(Native Method)
    at java.lang.invoke.MemberName$Factory.resolve(MemberName.java:975)
    at java.lang.invoke.MemberName$Factory.resolveOrFail(MemberName.java:1000)
    at java.lang.invoke.MethodHandles$Lookup.resolveOrFail(MethodHandles.java:1389)
    at java.lang.invoke.MethodHandles$Lookup.linkMethodHandleConstant(MethodHandles.java:1745)
    at java.lang.invoke.MethodHandleNatives.linkMethodHandleConstant(MethodHandleNatives.java:477)
    ... 26 more
Caused by: java.lang.ClassNotFoundException: com.tdunning.math.stats.TDigest
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 32 more

Scala version: 2.11.7 sbt version: 0.13.9 Elasticsearch Version: 5.4.0 Using play framework.

Please let me know if you need any more information. We were using elastic4s 5.0.1 before upgrade. I tried blowing up ivy cache, but no luck.

[sksamuel]

Try 5.4.2, but also check out the quick start samples: https://github.com/sksamuel/elastic4s/tree/master/samples

[animageofmine]

I just tried it without any luck, I get the same error. Did you test the latest code against Elasticsearch (not elastic4s) 5.4?

[sksamuel]

Yes, look at the sample projects. They all use 5.4.2.

[animageofmine]

I looked at sample projects. The one that applies in my case is tcp-client-sbt. The only difference I see is scala version 2.12. In my case, I use 2.11.7. From the above error, it does not seem like a problem.

I am trying to perform aggregations, which is not working. Most of the other queries work fine.

[sksamuel]

Ok but when you opened the ticket you said you only posted code creating the client. Can you post up a full example of what fails and then I can help.

[animageofmine]

BTW, moving back to elastic4s 5.3.2 fixes the problem.

Here is an example of aggregation query from my code. Please note that the debugger never reaches here:

trait AggregationsService
{
def esClient: TcpClient

def aggregate(request: AggregateRequest): Future[Either[ErrorInfo, AggregateResponse]] = 
{
    // debugger never reaches here
    val aggregateResponseFuture = esClient.execute{
      search(indexName)
        .size(0)
        .aggregations(getAggregateQuery(request))
        .query(constantScoreQuery(getFilterQuery(request).get))
    }
    // rest of the code
}

trait AggregationsModule  {
    lazy val aggregationService = new AggregationsService
    with ElasticsearchProfile
}

It fails here.

trait ElasticsearchProfile  {
  val settings = Settings.builder().put("cluster.name", ElasticsearchClusterName).build()
  // fails specifically here
  val esClient = TcpClient.transport(settings, ElasticsearchClientUri(ElasticsearchHost, ElasticsearchPort))
}

This is exactly where it fails (SearchModule.java):

        registerAggregation(new AggregationSpec(PercentilesAggregationBuilder.NAME, PercentilesAggregationBuilder::new,
                PercentilesAggregationBuilder::parse)
                    .addResultReader(InternalTDigestPercentiles.NAME, InternalTDigestPercentiles::new)
                    .addResultReader(InternalHDRPercentiles.NAME, InternalHDRPercentiles::new));

build.sbt imports the following:

libraryDependencies ++= Seq(
  "com.sksamuel.elastic4s" %% "elastic4s-core" % "5.4.2",
  "com.sksamuel.elastic4s" %% "elastic4s-tcp" % "5.4.2",
)

It looks like there is some library that is not explicitly imported to ivy cache, hence the code cannot find the class.

[sksamuel]

It's prob a dependency issue like you surmise, but I can't help unless you can make a stand alone code snippet I can paste. Things like AggregationSpec, registerAggregation don't exist in the code you just pasted so I'd have to guess what they mean.

[animageofmine]

I should be able to create a code snippet. Sorry, I couldn't paste production code base completely.

I am a curious though because the issue didn't hit you. Anyway, I should be able to get back with a code snippet later in the evening today (I am located in Pacific Time Zone).

[sksamuel]

I understand completely about production code. If you can post up a self contained snippet that fails, I'll fix it 100%.

[sksamuel]

I'm in UTC btw, so prob be tomorrow night now.

[animageofmine]

Thanks Stephen. I think this can wait for one night since I changed the code and moved it back to 5.3.2.

[animageofmine]

Following is the code snippet I used to repro the bug in a separate project.

Let me know if you need more info.

Elasticsearch version: 5.3.2 Elastic4s version: 5.4.2

I imported index from here: https://www.elastic.co/guide/en/elasticsearch/reference/current/_exploring_your_data.html

package controllers

import com.sksamuel.elastic4s.{ElasticsearchClientUri, TcpClient}
import org.elasticsearch.common.settings.Settings
import play.api.mvc.{Action, Controller}
import com.sksamuel.elastic4s.ElasticDsl._
import com.sksamuel.elastic4s.searches.{RichSearchResponse, SearchDefinition}
import org.elasticsearch.search.aggregations.bucket.terms.StringTerms
import org.elasticsearch.search.aggregations.bucket.terms.Terms.Bucket
import org.elasticsearch.search.aggregations.metrics.max.InternalMax

import scala.concurrent._
import ExecutionContext.Implicits.global

object Elastic4sController extends Controller {

  lazy val settings = Settings.builder().put("cluster.name", "localcluster").build()
  // This is where code throws exception. 
  lazy val esClient = TcpClient.transport(settings, ElasticsearchClientUri("elasticsearch://localhost:9300"))

  // Debugger never reaches here
  def aggregationTest = Action.async {
    esClient.execute {
      search("bank")
        .size(0)
        .aggregations(sumAgg("agebalance", "balance"))
    }.map {
      (response: RichSearchResponse) =>
        // Do something
        println(response.ids)
      Ok(views.html.index("successful"))
    }
  }

}

build.sbt

scalaVersion := "2.11.7"

libraryDependencies ++= Seq( jdbc , cache , ws   , specs2 % Test )

unmanagedResourceDirectories in Test <+=  baseDirectory ( _ /"target/web/public/test" )  

resolvers += "scalaz-bintray" at "https://dl.bintray.com/scalaz/releases"

resolvers += DefaultMavenRepository

resolvers += "Sonatype OSS" at "https://oss.sonatype.org/content/repositories/snapshots"

resolvers += "Sonatype Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots/"

resolvers += "Sonatype Releases" at "https://oss.sonatype.org/content/repositories/releases/"

libraryDependencies ++= Seq(
  "com.sksamuel.elastic4s" %% "elastic4s-core" % "5.4.2",
  "com.sksamuel.elastic4s" %% "elastic4s-tcp" % "5.4.2",
  "com.sksamuel.elastic4s" %% "elastic4s-http" % "5.4.2", 
)

build.properties

sbt.version=0.13.5

plugins.sbt

logLevel := Level.Warn

resolvers += "Typesafe repository" at "http://repo.typesafe.com/typesafe/releases/"

resolvers += "Typesafe Snapshots" at "http://repo.typesafe.com/typesafe/snapshots/"

addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.4.2")

[animageofmine]

@sksamuel any update?

[sksamuel]

I can't duplicate your issue.

I've taken your snippets and made a project out of it. What you pasted didn't compile as you had a dangling comma in the sbt file and I removed the Controller super class, as I have no idea how to run a play project, and replaced it with App so its runnable. https://github.com/sksamuel/elastic4s/tree/master/samples/issue907

I suspect your issue is that you have an older version of elastic lurking about on your classpath, and in that version the constructor it needs doesn't exist. Or just get rid of the TCP client and use the HTTP one exclusively.

[sksamuel]

Closing this as not heard otherwise.