samskivert / jmustache

A Java implementation of the Mustache templating language.
Other
834 stars 128 forks source link

Recursive template throws stack overflow #124

Closed spacether closed 3 years ago

spacether commented 3 years ago

Hi, I have a recursive template that throws a StackOverflow

In my case I am iterating over openapi specs and outputting a python class for each defined schema. A type object schema contains properties. Each of those properties is also a schema. Users may nest schema definition as deep as they want. Usually they only include the object and its schema properties. But sometimes the nest schema definitions much deeper.

M simplest use case is:

file schema.mustache

{{indent}}class  {{name}}:
{{indent}}    @property
{{indent}}    def properties():
{{#properties}}
{{> schema.mustache }}
{{/properties}}

indent stores the current indentation level string. In the first class indent is length 0 string In the properties indent is a string with 4 spaces in it In properties properties indent is a string with 8 spaces in it etc...

And when I have a template like this I get a StackOverflowError. Can this be fixed?

samskivert commented 3 years ago

Hrm, what is the stack overflow error you're getting? I just tried writing a test for a recursive partial and it seems to work fine:


    @Test public void testRecursivePartial () {
        String template = "[{{name}}{{#properties}}, {{> schema.mustache}}{{/properties}}]";
        test(Mustache.compiler().withLoader(new Mustache.TemplateLoader() {
            public Reader getTemplate (String name) {
                return new StringReader(template);
            }
        }), "[level0, [level1, [level2]]]", template,
             context("name", "level0",
                     "properties", context("name", "level1",
                                           "properties", context("name", "level2",
                                                                 "properties", null))));
    }

It expands the partial up to the point that it hits the null and then stops.

spacether commented 3 years ago
Exception in thread "main" java.lang.StackOverflowError
    at java.base/sun.nio.fs.UnixPath.encode(UnixPath.java:118)
    at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
    at java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
    at java.base/java.nio.file.Path.of(Path.java:147)
    at java.base/java.nio.file.Paths.get(Paths.java:69)
    at org.openapitools.codegen.templating.GeneratorTemplateContentLocator.buildLibraryFilePath(GeneratorTemplateContentLocator.java:27)
    at org.openapitools.codegen.templating.GeneratorTemplateContentLocator.getFullTemplatePath(GeneratorTemplateContentLocator.java:67)
    at org.openapitools.codegen.TemplateManager.lambda$getFullTemplateFile$0(TemplateManager.java:48)
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
    at java.base/java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:958)
    at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:127)
    at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:502)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:488)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
    at java.base/java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:150)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.base/java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:543)
    at org.openapitools.codegen.TemplateManager.getFullTemplateFile(TemplateManager.java:50)
    at org.openapitools.codegen.TemplateManager.getFullTemplateContents(TemplateManager.java:72)
    at org.openapitools.codegen.templating.MustacheEngineAdapter.findTemplate(MustacheEngineAdapter.java:68)
    at org.openapitools.codegen.templating.MustacheEngineAdapter.lambda$compileTemplate$0(MustacheEngineAdapter.java:57)
    at com.samskivert.mustache.Mustache$IncludedTemplateSegment.execute(Mustache.java:756)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:845)
    at com.samskivert.mustache.Mustache$InvertedSegment.execute(Mustache.java:910)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:845)
    at com.samskivert.mustache.Mustache$InvertedSegment.execute(Mustache.java:906)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:845)
    at com.samskivert.mustache.Mustache$SectionSegment.execute(Mustache.java:866)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:845)
    at com.samskivert.mustache.Mustache$SectionSegment.execute(Mustache.java:870)
    at com.samskivert.mustache.Template.executeSegs(Template.java:157)

model.mustache includes model_normal.mustache model_normal.mustache includes schema.mustache schema.mustache includes model_normal.mustache

and for reference the project that I am in uses jmustache 1.14

spacether commented 3 years ago

Just now I updated to jmustache v1.15 and it also has the stack overflow:

Exception in thread "main" java.lang.StackOverflowError
    at java.base/java.nio.ByteBuffer.limit(ByteBuffer.java:1099)
    at java.base/java.nio.ByteBuffer.limit(ByteBuffer.java:262)
    at java.base/java.nio.Buffer.<init>(Buffer.java:222)
    at java.base/java.nio.ByteBuffer.<init>(ByteBuffer.java:281)
    at java.base/java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:75)
    at java.base/java.nio.ByteBuffer.wrap(ByteBuffer.java:393)
    at java.base/java.nio.ByteBuffer.wrap(ByteBuffer.java:422)
    at java.base/sun.nio.fs.UnixPath.encode(UnixPath.java:133)
    at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
    at java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
    at java.base/java.nio.file.Path.of(Path.java:147)
    at java.base/java.nio.file.Paths.get(Paths.java:69)
    at org.openapitools.codegen.templating.GeneratorTemplateContentLocator.buildLibraryFilePath(GeneratorTemplateContentLocator.java:27)
    at org.openapitools.codegen.templating.GeneratorTemplateContentLocator.getFullTemplatePath(GeneratorTemplateContentLocator.java:67)
    at org.openapitools.codegen.TemplateManager.lambda$getFullTemplateFile$0(TemplateManager.java:48)
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
    at java.base/java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:958)
    at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:127)
    at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:502)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:488)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
    at java.base/java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:150)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.base/java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:543)
    at org.openapitools.codegen.TemplateManager.getFullTemplateFile(TemplateManager.java:50)
    at org.openapitools.codegen.TemplateManager.getFullTemplateContents(TemplateManager.java:72)
    at org.openapitools.codegen.templating.MustacheEngineAdapter.findTemplate(MustacheEngineAdapter.java:68)
    at org.openapitools.codegen.templating.MustacheEngineAdapter.lambda$compileTemplate$0(MustacheEngineAdapter.java:57)
    at com.samskivert.mustache.Mustache$Compiler.loadTemplate(Mustache.java:220)
    at com.samskivert.mustache.Mustache$IncludedTemplateSegment.getTemplate(Mustache.java:845)
    at com.samskivert.mustache.Mustache$IncludedTemplateSegment.execute(Mustache.java:831)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:920)
    at com.samskivert.mustache.Mustache$InvertedSegment.execute(Mustache.java:992)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:920)
    at com.samskivert.mustache.Mustache$InvertedSegment.execute(Mustache.java:988)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:920)
    at com.samskivert.mustache.Mustache$SectionSegment.execute(Mustache.java:941)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:920)
    at com.samskivert.mustache.Mustache$SectionSegment.execute(Mustache.java:945)
    at com.samskivert.mustache.Template.executeSegs(Template.java:170)
    at com.samskivert.mustache.Mustache$IncludedTemplateSegment.execute(Mustache.java:831)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:920)
    at com.samskivert.mustache.Mustache$SectionSegment.execute(Mustache.java:945)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:920)
    at com.samskivert.mustache.Mustache$InvertedSegment.execute(Mustache.java:988)
    at com.samskivert.mustache.Template.executeSegs(Template.java:170)
    at com.samskivert.mustache.Mustache$IncludedTemplateSegment.execute(Mustache.java:831)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:920)
    at com.samskivert.mustache.Mustache$InvertedSegment.execute(Mustache.java:992)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:920)
    at com.samskivert.mustache.Mustache$InvertedSegment.execute(Mustache.java:988)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:920)
    at com.samskivert.mustache.Mustache$SectionSegment.execute(Mustache.java:941)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:920)
    at com.samskivert.mustache.Mustache$SectionSegment.execute(Mustache.java:945)
    at com.samskivert.mustache.Template.executeSegs(Template.java:170)
    at com.samskivert.mustache.Mustache$IncludedTemplateSegment.execute(Mustache.java:831)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:920)
    at com.samskivert.mustache.Mustache$SectionSegment.execute(Mustache.java:945)
    at com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:920)
spacether commented 3 years ago

So this may have been happening because the data that we are ingesting at the deeper levels is missing the properties key. Because of that it looks like the outer context properties data is used for the partial I think leading to the stack overflow.

On my side I will open a PR to fix the input data issue. If that fixes it I will close this issue. Thank you for adding the recursive test case to jmustache.

spacether commented 3 years ago

Sorry for the false alarm. My recursion was caused by bad input data as I describe above. Your library handles this recursive use case correctly.