HubSpot / jinjava

Jinja template engine for Java
Apache License 2.0
708 stars 170 forks source link

[Solved] Jinjaja code runs in UT but fails when embedded in Tomcat ?? #762

Open dchiaramello opened 3 years ago

dchiaramello commented 3 years ago

Hello.

I am facing a weird exception (Could not find expression factory class) I don't have with Unit tests with the following template: "SVNFMGRANTID{{ vnfd_id }}{% if index is not none %}{{ index }}{% endif %}"

The Unit Test is successfully rendering the template, but when that code is in a tomee-based micro-service, it fails...

When logging the exception, I have the following TemplateError:

Corresponding Exception:

com.hubspot.jinjava.interpret.TemplateSyntaxException: Could not find expression factory class
    at com.hubspot.jinjava.el.ExpressionResolver.resolveExpression(ExpressionResolver.java:194)
    at com.hubspot.jinjava.interpret.JinjavaInterpreter.resolveELExpression(JinjavaInterpreter.java:577)
    at com.hubspot.jinjava.interpret.JinjavaInterpreter.resolveELExpression(JinjavaInterpreter.java:590)
    at com.hubspot.jinjava.lib.tag.IfTag.isPositiveIfElseNode(IfTag.java:133)
    at com.hubspot.jinjava.lib.tag.IfTag.interpret(IfTag.java:91)
    at com.hubspot.jinjava.lib.tag.Tag.interpretOutput(Tag.java:27)
    at com.hubspot.jinjava.tree.TagNode.render(TagNode.java:54)
    at com.hubspot.jinjava.interpret.JinjavaInterpreter.render(JinjavaInterpreter.java:280)
    at com.hubspot.jinjava.interpret.JinjavaInterpreter.render(JinjavaInterpreter.java:228)
    at com.hubspot.jinjava.Jinjava.renderForResult(Jinjava.java:252)
    at com.hubspot.jinjava.Jinjava.renderForResult(Jinjava.java:208)

I suspect that the interesting thing may be fieldName='index is not none', I find it strange that it looks for a fieldName with that value - could it be possible that it failed to parse the template?

It may be something similar to the Filters don't seem to work issue, solved by excluding javassist and javax.el, but I don't have these dependencies in my project:

[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ nfv-svnfm ---
[INFO] com.gemalto.telecom:nfv-svnfm:war:7.6.0-SNAPSHOT
[INFO] +- javax:javaee-api:jar:8.0:provided
[INFO] +- org.apache.cxf:cxf-core:jar:3.2.7:provided
[INFO] +- org.apache.cxf:cxf-rt-transports-http:jar:3.2.7:provided
[INFO] +- org.apache.cxf:cxf-rt-rs-client:jar:3.2.7:provided
[INFO] |  \- org.apache.cxf:cxf-rt-frontend-jaxrs:jar:3.2.7:provided
[INFO] |     \- javax.annotation:javax.annotation-api:jar:1.3:compile
[INFO] +- org.bouncycastle:bcpkix-jdk14:jar:1.65:compile
[INFO] +- org.bouncycastle:bcprov-jdk14:jar:1.65:compile
[INFO] +- org.yaml:snakeyaml:jar:1.23:compile
[INFO] +- org.apache.logging.log4j:log4j-api:jar:2.13.1:compile
[INFO] +- com.gemalto.telecom.ota:ota-core-lib:jar:20210910-100956:compile
[INFO] |  +- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:2.10.5:compile
[INFO] |  +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.10.5:compile
[INFO] |  +- com.google.guava:guava:jar:25.1-jre:compile
[INFO] |  +- io.dropwizard.metrics:metrics-healthchecks:jar:3.2.6:compile
[INFO] |  +- io.dropwizard.metrics:metrics-jvm:jar:3.2.6:compile
[INFO] |  +- io.dropwizard.metrics:metrics-servlets:jar:3.2.6:compile
[INFO] |  |  +- io.dropwizard.metrics:metrics-json:jar:3.2.6:compile
[INFO] |  |  \- com.papertrail:profiler:jar:1.0.2:compile
[INFO] |  |     \- joda-time:joda-time:jar:2.9.1:compile
[INFO] |  +- io.prometheus:simpleclient:jar:0.7.0:compile
[INFO] |  \- io.prometheus:simpleclient_servlet:jar:0.7.0:compile
[INFO] |     \- io.prometheus:simpleclient_common:jar:0.7.0:compile
[INFO] +- com.gemalto.rnd.logging:unified-log-format:jar:5.5.2:compile
[INFO] |  +- org.apache.logging.log4j:log4j-core:jar:2.13.1:compile
[INFO] |  +- org.apache.logging.log4j:log4j-1.2-api:jar:2.13.1:compile
[INFO] |  +- org.apache.logging.log4j:log4j-slf4j-impl:jar:2.13.1:compile
[INFO] |  +- org.apache.logging.log4j:log4j-web:jar:2.13.1:compile
[INFO] |  +- org.apache.logging.log4j:log4j-jul:jar:2.13.1:compile
[INFO] |  +- com.lmax:disruptor:jar:3.4.2:compile
[INFO] |  \- org.slf4j:slf4j-api:jar:1.7.30:compile
[INFO] +- io.dropwizard.metrics:metrics-core:jar:3.2.6:compile
[INFO] +- com.datastax.oss:java-driver-core:jar:4.13.0:compile
[INFO] |  +- com.datastax.oss:native-protocol:jar:1.5.0:compile
[INFO] |  +- io.netty:netty-handler:jar:4.1.42.Final:compile
[INFO] |  |  +- io.netty:netty-common:jar:4.1.42.Final:compile
[INFO] |  |  +- io.netty:netty-buffer:jar:4.1.42.Final:compile
[INFO] |  |  +- io.netty:netty-transport:jar:4.1.42.Final:compile
[INFO] |  |  |  \- io.netty:netty-resolver:jar:4.1.42.Final:compile
[INFO] |  |  \- io.netty:netty-codec:jar:4.1.42.Final:compile
[INFO] |  +- com.datastax.oss:java-driver-shaded-guava:jar:25.1-jre-graal-sub-1:compile
[INFO] |  +- com.typesafe:config:jar:1.4.1:compile
[INFO] |  +- com.github.jnr:jnr-posix:jar:3.1.5:compile
[INFO] |  |  +- com.github.jnr:jnr-ffi:jar:2.2.2:compile
[INFO] |  |  |  +- com.github.jnr:jffi:jar:1.3.1:compile
[INFO] |  |  |  +- com.github.jnr:jffi:jar:native:1.3.1:runtime
[INFO] |  |  |  +- org.ow2.asm:asm:jar:9.1:compile
[INFO] |  |  |  +- org.ow2.asm:asm-commons:jar:9.1:compile
[INFO] |  |  |  +- org.ow2.asm:asm-analysis:jar:9.1:compile
[INFO] |  |  |  +- org.ow2.asm:asm-tree:jar:9.1:compile
[INFO] |  |  |  +- org.ow2.asm:asm-util:jar:9.1:compile
[INFO] |  |  |  +- com.github.jnr:jnr-a64asm:jar:1.0.0:compile
[INFO] |  |  |  \- com.github.jnr:jnr-x86asm:jar:1.0.2:compile
[INFO] |  |  \- com.github.jnr:jnr-constants:jar:0.10.1:compile
[INFO] |  +- org.hdrhistogram:HdrHistogram:jar:2.1.12:compile
[INFO] |  +- com.esri.geometry:esri-geometry-api:jar:1.2.1:compile
[INFO] |  |  +- org.json:json:jar:20090211:compile
[INFO] |  |  \- org.codehaus.jackson:jackson-core-asl:jar:1.9.12:compile
[INFO] |  +- org.reactivestreams:reactive-streams:jar:1.0.3:compile
[INFO] |  +- com.github.stephenc.jcip:jcip-annotations:jar:1.0-1:compile
[INFO] |  \- com.github.spotbugs:spotbugs-annotations:jar:3.1.12:compile
[INFO] |     \- com.google.code.findbugs:jsr305:jar:3.0.2:compile
[INFO] +- com.datastax.oss:java-driver-query-builder:jar:4.13.0:compile
[INFO] +- com.fasterxml.jackson.core:jackson-annotations:jar:2.11.2:compile
[INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.11.2:compile
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.11.2:compile
[INFO] +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.11.2:compile
[INFO] +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:jar:2.11.2:compile
[INFO] |  +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:jar:2.11.2:compile
[INFO] |  \- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.11.2:compile
[INFO] |     +- jakarta.xml.bind:jakarta.xml.bind-api:jar:2.3.2:compile
[INFO] |     \- jakarta.activation:jakarta.activation-api:jar:1.2.1:compile
[INFO] +- org.glassfish.jersey.core:jersey-client:jar:2.25.1:compile
[INFO] |  +- javax.ws.rs:javax.ws.rs-api:jar:2.0.1:compile
[INFO] |  +- org.glassfish.jersey.core:jersey-common:jar:2.25.1:compile
[INFO] |  |  +- org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.25.1:compile
[INFO] |  |  \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:compile
[INFO] |  +- org.glassfish.hk2:hk2-api:jar:2.5.0-b32:compile
[INFO] |  |  +- org.glassfish.hk2:hk2-utils:jar:2.5.0-b32:compile
[INFO] |  |  \- org.glassfish.hk2.external:aopalliance-repackaged:jar:2.5.0-b32:compile
[INFO] |  +- org.glassfish.hk2.external:javax.inject:jar:2.5.0-b32:compile
[INFO] |  \- org.glassfish.hk2:hk2-locator:jar:2.5.0-b32:compile
[INFO] +- com.googlecode.json-simple:json-simple:jar:1.1.1:compile
[INFO] +- io.swagger:swagger-annotations:jar:1.5.23:compile
[INFO] +- com.google.code.gson:gson:jar:2.8.5:compile
[INFO] +- com.github.spullara.mustache.java:compiler:jar:0.9.5:compile
[INFO] +- com.hubspot.jinjava:jinjava:jar:2.5.10:compile
[INFO] |  +- org.javassist:javassist:jar:3.24.1-GA:compile
[INFO] |  +- org.jsoup:jsoup:jar:1.10.3:compile
[INFO] |  +- com.google.re2j:re2j:jar:1.2:compile
[INFO] |  +- org.apache.commons:commons-lang3:jar:3.9:compile
[INFO] |  +- commons-net:commons-net:jar:3.3:compile
[INFO] |  +- com.googlecode.java-ipv6:java-ipv6:jar:0.17:compile
[INFO] |  +- com.google.code.findbugs:annotations:jar:3.0.1:compile
[INFO] |  \- ch.obermuhlner:big-math:jar:2.0.0:compile
[INFO] +- junit:junit:jar:4.12:test
[INFO] |  \- org.hamcrest:hamcrest-core:jar:1.3:test
[INFO] +- com.xebialabs.restito:restito:jar:0.9.3:test
[INFO] |  +- com.jayway.jsonpath:json-path:jar:2.1.0:test
[INFO] |  |  \- net.minidev:json-smart:jar:2.2:test
[INFO] |  |     \- net.minidev:accessors-smart:jar:1.1:test
[INFO] |  \- org.apache.mina:mina-core:jar:2.0.13:test
[INFO] +- org.glassfish.grizzly:grizzly-http:jar:2.3.25:test
[INFO] +- org.glassfish.grizzly:grizzly-framework:jar:2.3.25:test
[INFO] +- org.junit.jupiter:junit-jupiter-api:jar:5.4.2:test
[INFO] |  +- org.apiguardian:apiguardian-api:jar:1.0.0:test
[INFO] |  +- org.opentest4j:opentest4j:jar:1.1.1:test
[INFO] |  \- org.junit.platform:junit-platform-commons:jar:1.4.2:test
[INFO] +- org.mockito:mockito-core:jar:2.25.1:test
[INFO] |  +- net.bytebuddy:byte-buddy:jar:1.9.7:test
[INFO] |  +- net.bytebuddy:byte-buddy-agent:jar:1.9.7:test
[INFO] |  \- org.objenesis:objenesis:jar:2.6:test
[INFO] +- org.glassfish.grizzly:grizzly-http-server:jar:2.3.25:test
[INFO] +- org.jacoco:org.jacoco.agent:jar:runtime:0.8.5:test
[INFO] +- com.gemalto.telecom.nfv:nfv-simulator-api:jar:1.0.0:test
[INFO] |  +- org.springframework.boot:spring-boot-starter-web:jar:2.1.0.RELEASE:test
[INFO] |  |  +- org.springframework.boot:spring-boot-starter:jar:2.1.0.RELEASE:test
[INFO] |  |  |  +- org.springframework.boot:spring-boot:jar:2.1.0.RELEASE:test
[INFO] |  |  |  +- org.springframework.boot:spring-boot-autoconfigure:jar:2.1.0.RELEASE:test
[INFO] |  |  |  \- org.springframework:spring-core:jar:5.1.2.RELEASE:test
[INFO] |  |  |     \- org.springframework:spring-jcl:jar:5.1.2.RELEASE:test
[INFO] |  |  +- org.springframework.boot:spring-boot-starter-json:jar:2.1.0.RELEASE:test
[INFO] |  |  |  \- com.fasterxml.jackson.module:jackson-module-parameter-names:jar:2.9.7:test
[INFO] |  |  +- org.springframework.boot:spring-boot-starter-tomcat:jar:2.1.0.RELEASE:test
[INFO] |  |  |  +- org.apache.tomcat.embed:tomcat-embed-core:jar:9.0.12:test
[INFO] |  |  |  +- org.apache.tomcat.embed:tomcat-embed-el:jar:9.0.12:test
[INFO] |  |  |  \- org.apache.tomcat.embed:tomcat-embed-websocket:jar:9.0.12:test
[INFO] |  |  +- org.hibernate.validator:hibernate-validator:jar:6.0.13.Final:test
[INFO] |  |  |  +- javax.validation:validation-api:jar:2.0.1.Final:test
[INFO] |  |  |  +- org.jboss.logging:jboss-logging:jar:3.3.2.Final:test
[INFO] |  |  |  \- com.fasterxml:classmate:jar:1.3.4:test
[INFO] |  |  +- org.springframework:spring-web:jar:5.1.2.RELEASE:test
[INFO] |  |  |  \- org.springframework:spring-beans:jar:5.1.2.RELEASE:test
[INFO] |  |  \- org.springframework:spring-webmvc:jar:5.1.2.RELEASE:test
[INFO] |  |     +- org.springframework:spring-aop:jar:5.1.2.RELEASE:test
[INFO] |  |     +- org.springframework:spring-context:jar:5.1.2.RELEASE:test
[INFO] |  |     \- org.springframework:spring-expression:jar:5.1.2.RELEASE:test
[INFO] |  +- org.springframework.boot:spring-boot-starter-log4j2:jar:2.1.0.RELEASE:test
[INFO] |  |  \- org.slf4j:jul-to-slf4j:jar:1.7.25:test
[INFO] |  \- commons-io:commons-io:jar:2.6:test
[INFO] +- org.apache.httpcomponents:httpclient:jar:4.5.6:compile
[INFO] |  +- commons-logging:commons-logging:jar:1.2:compile
[INFO] |  \- commons-codec:commons-codec:jar:1.11:compile
[INFO] \- org.apache.httpcomponents:httpcore:jar:4.4.11:compile

Would you have any idea/suggestion, please?

Thanks for your attention, Daniel

dchiaramello commented 3 years ago

Hello.

I eventually solved it by setting the Classloader when using Jinjava:

ClassLoader curClassLoader = Thread.currentThread().getContextClassLoader();
try {
    Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());

    <Jinjava code there>
} finally {
    Thread.currentThread().setContextClassLoader(curClassLoader);
}

Regards, Daniel

smoomrik commented 2 years ago

Should only initialization of jinjava be wrapped or in addition every call to render?

dchiaramello commented 2 years ago

I have to do it for both the initialization and the render calls...

smoomrik commented 2 years ago

Few words about the case that I faced with. I have the following in code:

CompletableFuture.supplyAsync(() -> some work)
.thenAcceptAsync( a part of code that use jinjava, executor)
... 

So what I observe in this case, the supply async uses fork join pool for spawning threads at executor where jinjava is used. That means these threads will have system classloader that will not work at jinjava. Another one point, if work submitted inside supplyAsync completes before chain is constructed, threads at passed executor will be spawned by the main thread and in this case it will work fine.

I would not say its solved by fix provided above as usually developers doesn't track the classloaders picked. It leads to huge amount of time spent on triagging such kind of issues. It would be nice to see a fix from jinjava side.