Conflicting jar versions can cause plugins not to work properly

[purbon]

Raised becuase of a conflict with lucene versions between elasticsearch and neo4j, the test raised the situation where having two conflicting versions cause the plugins not to register.

To reproduce this issue:

1.- Install logstash-input-neo4j. 2.- Install logstash-output-elasticsearch. 3.- Add development dependencies. 4.- Run the plugin test.

then you will get an error like:

Failures:

  1) outputs/elasticsearch registration should register
     Failure/Error: expect {output.register}.to_not raise_error
       expected no Exception, got java.lang.NoSuchFieldError: LUCENE_3_6 with backtrace:
         # org.elasticsearch.Version.<clinit>(org/elasticsearch/Version.java:43)
         # org.elasticsearch.client.transport.TransportClient.<init>(org/elasticsearch/client/transport/TransportClient.java:168)
         # org.elasticsearch.client.transport.TransportClient.<init>(org/elasticsearch/client/transport/TransportClient.java:123)
         # java.lang.reflect.Constructor.newInstance(java/lang/reflect/Constructor.java:526)
         # RUBY.(root)(./rakelib/test.rake:38)
     # ./rakelib/test.rake:38:in `(root)'

Finished in 0.967 seconds
21 examples, 1 failure

but if for example you install the logstash-output-neo4j and the logstash-output-elasticsearch, the neo4j plugin output an error as it can has a conflict with the lucene version.

lucene jars being used are:

.//bundle/jruby/1.9/gems/logstash-output-elasticsearch-0.2.4-java/vendor/jar-dependencies/runtime-jars/lucene-core-4.10.4.jar
.//bundle/jruby/1.9/gems/neo4j-community-2.0.1/lib/neo4j-community/jars/lucene-core-3.6.2.jar

This happen now with Lucene, but I guess it can happen with other dependencies alike in the feature too.

[jsvd]

have you tried using es's version of lucene in neo4j plugin? we did a similar thing for logstash-output-elasticsearch and the amazon aws plugin. or maybe there's a neo4j plugin that will bring lucene 4?

[suyograo]

test notification

[purbon]

For now I see some ways to get this fix:

• Have custom class loaders that load each plugin dependencies. This might require some work as jruby depends on a single class loader per runtime. [Still under research, but hardly the solution for this nowadays].
• To do some jar shading. This in my opinion is an ungly workaround that require some post processing of the dependencies to be included in here.
• To basically update the dependencies internally of the plugins. In this case (Neo4j/Elasticsearch) is not possible as boths are in different branches and Neo4j can not be updated easily.
• To have plugins that, because of their dependencies, don't work well with others.

@suyograo @jsvd @ph ideas? opinions.

[jsvd]

Can you explain a bit more the "jar shading"? I'm not familiar with it.

Whenever possible, updating the dependencies is the strategy will less friction and less problems. It is not applicable in the neo4j+es situation however..

The class loaders scenario is ideal, but will require quite some investigation, maybe something that should be addressed with the help of the JRuby team?

[colinsurprenant]

In the java-world the preferred method for avoiding/dealing/preventing such "unresolvable" dependencies conflicts is to use jar-shading. It is quite widely adopted. Not sure how we could use that in our context but I would investigate.

I don't think it's even possible to deal with that at the Ruby level, it's a Java issue at the JVM level, Java classes are conflicting.

[colinsurprenant]

So, one solution involving only the end that we control would be to create a shaded elasticsearch uber-jar for usage in the plugin(s).

[purbon]

Hi, i see pros and cons of each of this solutions, I will try to enumerate them so we are able to have an informative decision:

1.- ClassLoader approach: This nice as it involve _no change on the jars used as dependencies_, however it's _going to be hard_ as JRuby works with one classloader. The only way I've seen, actually in trinidad to do something smilar is by having multiple runtimes. This might _require some work on the JRuby_ side.

2.- Jar-Shading: This _is good_, as for now the situation is with a project and dependencies we can control. We could do a new jar with all dependencies shades for ES. However this will make _packaging more complicated_ (we should do the shading automatic every time). Not sure if we can easy do the shading with dependencies we don't control.

A good solution for this problem would be a solution that involved no change in the jars used as dependencies and that is trasparent to the end user, isn't?

My take for now would be that we:

1.- Trying to update the jar-dependencies not to colide whenever possible. (as we already did for some plugins). 2.- When not possible, add a note that say this two plugins don't work well together. 3.- Investigate if we can automate the jar-shading, so we got somehow this fix.

If we solve 3 in a nice way, we're good to go.

Would be nice in the long run not to need any jar-shading if possible, so even if we go with jar-shading I would not completely ignore other approaches.

Makes sense?

[purbon]

After giving it more thought, I'm more on the side of not doing jar shading, as the generate jar would not have enought/any testing, and this jar is moslty for the embedded use case (not much of it, right?) ...

To workaround this for now, I think we should:

• Not load all ES jars when not extrictly necessary.
• If case there is a complic we can not solve by other means, we simple say is known problem.

/cheers

[colinsurprenant]

Without having done deeper research & experimentation, I thing that solution (1) trying to modify class loading will be seriously difficult. Having multiple runtimes is, well, nowhere near easy to figure out correctly either. This is why shading is a "popular" solution because modifying class loading behaviour is usually just looking for troubles.

I think that the only practical short-term solution is to avoid those conflicts by not loading "known" conflicting dependencies, for now.

For the shading, obviously, if we decide to do that, it will have to be automated at build time, that should not be a "problem". And it should also not be a problem for including 3rd party jars in that shaded jar, that's what it's for. There is something I am obviously missing if it's a problem to do that with ES.