search

apache / druid

Apache Druid: a high performance real-time analytics database.
https://druid.apache.org/
Apache License 2.0
13.33k stars 3.66k forks source link

Facing issues in running Druid Indexer Job on Dataproc(GCP) #15646

Open ishanaj opened 6 months ago

ishanaj commented 6 months ago

I am trying to run Druid indexer job on Dataproc(GCP), but facing issue with javax-el api

Druid version used: 28.0.0 Dataproc runs on Java 11

INFO: An exception was caught and reported. Message: java.lang.NoSuchMethodError: 'javax.el.ExpressionFactory javax.el.ExpressionFactory.newInstance()'
javax.validation.ValidationException: HV000183: Unable to initialize 'javax.el.ExpressionFactory'. Check that you have the EL dependencies on the classpath, or use ParameterMessageInterpolator instead
    at org.hibernate.validator.messageinterpolation.ResourceBundleMessageInterpolator.buildExpressionFactory(ResourceBundleMessageInterpolator.java:102)
    at org.hibernate.validator.messageinterpolation.ResourceBundleMessageInterpolator.<init>(ResourceBundleMessageInterpolator.java:45)
    at org.hibernate.validator.internal.engine.ConfigurationImpl.getDefaultMessageInterpolator(ConfigurationImpl.java:423)
    at org.hibernate.validator.internal.engine.ConfigurationImpl.getDefaultMessageInterpolatorConfiguredWithClassLoader(ConfigurationImpl.java:575)
    at org.hibernate.validator.internal.engine.ConfigurationImpl.getMessageInterpolator(ConfigurationImpl.java:364)
    at org.hibernate.validator.internal.engine.ValidatorFactoryImpl.<init>(ValidatorFactoryImpl.java:144)
    at org.hibernate.validator.HibernateValidator.buildValidatorFactory(HibernateValidator.java:38)
    at org.hibernate.validator.internal.engine.ConfigurationImpl.buildValidatorFactory(ConfigurationImpl.java:331)
    at javax.validation.Validation.buildDefaultValidatorFactory(Validation.java:110)
    at org.apache.druid.guice.ConfigModule.configure(ConfigModule.java:39)
    at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
    at com.google.inject.spi.Elements.getElements(Elements.java:110)
    at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
    at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
    at com.google.inject.Guice.createInjector(Guice.java:96)
    at com.google.inject.Guice.createInjector(Guice.java:73)
    at org.apache.druid.guice.BaseInjectorBuilder.build(BaseInjectorBuilder.java:66)
    at org.apache.druid.cli.Main.main(Main.java:96)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at com.google.cloud.hadoop.services.agent.job.shim.HadoopRunClassShim.main(HadoopRunClassShim.java:19)
Caused by: java.lang.NoSuchMethodError: 'javax.el.ExpressionFactory javax.el.ExpressionFactory.newInstance()'
    at org.hibernate.validator.messageinterpolation.ResourceBundleMessageInterpolator.buildExpressionFactory(ResourceBundleMessageInterpolator.java:98)
    ... 22 more

Is something wrong with my Druid setup? Need some help in resolving this.

abhishekagarwal87 commented 6 months ago

You may want to set these job properties in your ingestion spec

"jobProperties": {
        "mapreduce.job.classloader": "true",
        "mapreduce.job.classloader.system.classes": "-javax.el.,org.apache.commons.logging.,org.apache.log4j.,org.apache.hadoop.,com.google.cloud.hadoop.fs.gcs.",
        "mapreduce.framework.name": "yarn"
      },