search

apache / incubator-stormcrawler

A scalable, mature and versatile web crawler based on Apache Storm
https://stormcrawler.apache.org/
Apache License 2.0
887 stars 262 forks source link

Upgrade dependency Storm 2.6.4 #1323

Closed mvolikas closed 2 weeks ago

mvolikas commented 3 weeks ago

The maven archetype pom has storm.version 2.6.2. When using this with stormcrawler-solr dependency a jetty version incompatibility issue arises. In particular, when submitting the shaded jar to Storm I get:

java.lang.IncompatibleClassChangeError:
class org.eclipse.jetty.http.HttpFields$Mutable can not implement org.eclipse.jetty.http.HttpFields,
because it is not an interface (org.eclipse.jetty.http.HttpFields is in unnamed module of loader 'app')

This is because Solr 9.6.1 uses Jetty 10. while Storm 2.6.2 Jetty 9.. Am I missing something? When switching to storm 2.6.4 this gets resolved. Given that in storm-crawler 3.1.0 Storm 2.6.4 is used in the main pom, maybe we could update the archetype pom to the latest version as well?

rzo1 commented 3 weeks ago

Yes. It was missed in the update.

jnioche commented 3 weeks ago

@mvolikas if you write a PR for it, can you please also add a fix for the core archetype ? Thanks

mvolikas commented 3 weeks ago

@jnioche I cannot open the link you gave. I guess you mean this pom right? This and the OpenSearch pom are the only places that have version 2.6.2.

jnioche commented 3 weeks ago

@jnioche I cannot open the link you gave. I guess you mean this pom right? This and the OpenSearch pom are the only places that have version 2.6.2.

yes, not sure where that link came from Thanks!