search

karatelabs / karate

Test Automation Made Simple
https://karatelabs.github.io/karate
MIT License
8.33k stars 1.95k forks source link

Scenario Outline with huge Examples amount produces StackOverflowError (v0.9.1) #661

Closed backfloep closed 5 years ago

backfloep commented 5 years ago

Hi Peter,

I recently upgraded from v0.8.0.1 to v0.9.1 and now one of my scenario outlines (Examples ~6400 rows) result in an StackOverflowError. 

Exception in thread "main" java.lang.StackOverflowError
    at java.lang.String.indexOf(String.java:1503)
    at java.net.URLStreamHandler.parseURL(URLStreamHandler.java:143)
    at sun.net.www.protocol.file.Handler.parseURL(Handler.java:67)
    at java.net.URL.<init>(URL.java:622)
    at java.net.URL.<init>(URL.java:490)
    at sun.misc.URLClassPath$FileLoader.getResource(URLClassPath.java:1259)
    at sun.misc.URLClassPath$FileLoader.findResource(URLClassPath.java:1249)
    at sun.misc.URLClassPath.findResource(URLClassPath.java:215)
    at java.net.URLClassLoader$2.run(URLClassLoader.java:569)
    at java.net.URLClassLoader$2.run(URLClassLoader.java:567)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findResource(URLClassLoader.java:566)
    at java.lang.ClassLoader.getResource(ClassLoader.java:1096)
    at com.intuit.karate.FileUtils.fromRelativeClassPath(FileUtils.java:399)
    at com.intuit.karate.FileUtils.toResource(FileUtils.java:162)
    at com.intuit.karate.FileUtils.getFileStream(FileUtils.java:195)
    at com.intuit.karate.FileUtils.readFileAsString(FileUtils.java:185)
    at com.intuit.karate.FileUtils.readFile(FileUtils.java:107)
    at com.intuit.karate.core.ScriptBridge.read(ScriptBridge.java:81)
    at jdk.nashorn.internal.scripts.Script$Recompilation$6$14A$\^eval\_.L:1(<eval>:1)
    at jdk.nashorn.internal.scripts.Script$5$\^eval\_.:program(<eval>:1)
    at jdk.nashorn.internal.runtime.ScriptFunctionData.invoke(ScriptFunctionData.java:637)
    at jdk.nashorn.internal.runtime.ScriptFunction.invoke(ScriptFunction.java:494)
    at jdk.nashorn.internal.runtime.ScriptRuntime.apply(ScriptRuntime.java:393)
    at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:449)
    at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:406)
    at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:402)
    at jdk.nashorn.api.scripting.NashornScriptEngine.eval(NashornScriptEngine.java:155)
    at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
    at com.intuit.karate.ScriptBindings.eval(ScriptBindings.java:147)
    at com.intuit.karate.ScriptBindings.updateBindingsAndEval(ScriptBindings.java:142)
    at com.intuit.karate.ScriptBindings.evalInNashorn(ScriptBindings.java:127)
    at com.intuit.karate.Script.evalJsExpression(Script.java:423)
    at com.intuit.karate.Script.evalKarateExpression(Script.java:337)
    at com.intuit.karate.Script.evalKarateExpression(Script.java:203)
    at com.intuit.karate.Script.call(Script.java:1621)
    at com.intuit.karate.Script.callAndUpdateConfigAndAlsoVarsIfMapReturned(Script.java:1773)
    at com.intuit.karate.core.ScenarioContext.<init>(ScenarioContext.java:237)
    at com.intuit.karate.StepActions.<init>(StepActions.java:52)
    at com.intuit.karate.core.ScenarioExecutionUnit.init(ScenarioExecutionUnit.java:105)
    at com.intuit.karate.core.ScenarioExecutionUnit.run(ScenarioExecutionUnit.java:179)
    at com.intuit.karate.core.FeatureExecutionUnit.lambda$new$1(FeatureExecutionUnit.java:51)
    at com.intuit.karate.core.FeatureExecutionUnit.run(FeatureExecutionUnit.java:123)
    at com.intuit.karate.core.FeatureExecutionUnit.lambda$new$1(FeatureExecutionUnit.java:51)
    at com.intuit.karate.core.FeatureExecutionUnit.lambda$run$2(FeatureExecutionUnit.java:116)
    at com.intuit.karate.core.ScenarioExecutionUnit.run(ScenarioExecutionUnit.java:191)
    at com.intuit.karate.core.ScenarioExecutionUnit.lambda$new$0(ScenarioExecutionUnit.java:66)
    at com.intuit.karate.core.ScenarioExecutionUnit.run(ScenarioExecutionUnit.java:187)
    at com.intuit.karate.core.ScenarioExecutionUnit.lambda$new$0(ScenarioExecutionUnit.java:66)
    at com.intuit.karate.core.ScenarioExecutionUnit.run(ScenarioExecutionUnit.java:187)
    at com.intuit.karate.core.ScenarioExecutionUnit.lambda$new$0(ScenarioExecutionUnit.java:66)
    at com.intuit.karate.core.ScenarioExecutionUnit.run(ScenarioExecutionUnit.java:187)
    at com.intuit.karate.core.ScenarioExecutionUnit.lambda$new$0(ScenarioExecutionUnit.java:66)
    at com.intuit.karate.core.ScenarioExecutionUnit.run(ScenarioExecutionUnit.java:187)
    at com.intuit.karate.core.ScenarioExecutionUnit.lambda$new$0(ScenarioExecutionUnit.java:66)
    at com.intuit.karate.core.ScenarioExecutionUnit.run(ScenarioExecutionUnit.java:187)
    at com.intuit.karate.core.ScenarioExecutionUnit.lambda$new$0(ScenarioExecutionUnit.java:66)
    at com.intuit.karate.core.ScenarioExecutionUnit.run(ScenarioExecutionUnit.java:187)
    at com.intuit.karate.core.ScenarioExecutionUnit.lambda$new$0(ScenarioExecutionUnit.java:66)
    at com.intuit.karate.core.ScenarioExecutionUnit.run(ScenarioExecutionUnit.java:187)
    at com.intuit.karate.core.ScenarioExecutionUnit.lambda$new$0(ScenarioExecutionUnit.java:66)
    at com.intuit.karate.core.ScenarioExecutionUnit.run(ScenarioExecutionUnit.java:187)
    at com.intuit.karate.core.FeatureExecutionUnit.lambda$new$1(FeatureExecutionUnit.java:51)
    at com.intuit.karate.core.FeatureExecutionUnit.run(FeatureExecutionUnit.java:123)
        ...

I attached a maven skeleton project (bugtest.zip) including a feature where the error can be easily reproduced. The example scenario outline doesn't do anything beside assigning the row values of the example table to variables in the scenario.

@parallel=false
Feature: StackOverflow Test

  Scenario Outline: Produce StackOverflow

    * def col1 = <test1>
    * def col2 = <test2>
    * def col3 = <test3>
    * def col4 = <test4>
    * def col5 = <test5>
    * def col6 = <test6>
    * def col7 = <test7>

    * print col1

    Examples:
      | test1  | test2 | test3  | test4  | test5 | test6 | test7 |
      |'ADD'| 'ADD'| 'ADD'| 'ADD'| 'ADD'| 'MODIFY'| 'invalid'|
      |'ADD'| 'ADD'| 'ADD'| 'ADD'| 'ADD'| 'UNMODIFIED'| 'invalid'|
      |'ADD'| 'ADD'| 'ADD'| 'ADD'| 'ADD'| 'DELETE'| 'invalid'|
      |'ADD'| 'ADD'| 'ADD'| 'ADD'| 'MODIFY'| 'ADD'| 'invalid'|
      ...and so on...

This Error does not occure with v0.8.0.1. Please let me know if you need further information.

Keep up the good work, Peter!

ptrthomas commented 5 years ago

@backfloep OMG

this may take a while :| meanwhile a suggestion, it may be an interesting experiment to generate these permutations programmatically or at least read them off a CSV file: https://github.com/intuit/karate#dynamic-scenario-outline

backfloep commented 5 years ago

@ptrthomas Thanks, for the advice! I'll check and let you know the result.

ptrthomas commented 5 years ago

@backfloep yes, you most likely will have the same stack overflow problem though. but would appreciate some stress-testing of the "dynamic scenario outlines" - note there was a fix for logging ref #660 - there are instructions to build from source in the wiki if you are interested

ptrthomas commented 5 years ago

@backfloep fixed in develop would be great if you build from source, test and confirm and also look at using the dynamic scenario outlines so you can provide feedback on that for large numbers of rows etc

backfloep commented 5 years ago

@ptrthomas Excuse the stupid question but how do I exactly build and integrate the sources?

I cloned the developement branch and built the jars (with the mvn jar job) and importeted karate-apache-1.0.0.jar, karate-junit4-1.0.0.jar and karate-core-1.0.0.jar project.

But how do I get the compile and test dependencies? Do I have to import them manually like this?

dependencies {
    testCompile fileTree(dir: 'libs', include: '*.jar')
    compile 'info.picocli:picocli:3.9.2'
    compile group: 'org.slf4j', name: 'jcl-over-slf4j', version: '1.7.25'
    testCompile group: 'org.slf4j', name: 'slf4j-jdk14', version: '1.7.25'
    compile group: 'org.slf4j', name: 'slf4j-api', version: '1.7.25'
    compile group: 'info.picocli', name: 'picocli', version: '3.0.1'
}

Right now when I want to run a feature within IntelliJ I get following error:

Testing started at 13:14 ...
/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/bin/java -Dorg.jetbrains.run.directory=/Users/fb/Downloads/bugtestorino/src/test/java "-javaagent:/private/var/folders/0j/38__jzzd1x30yf1srqhhqsl40000gn/T/AppTranslocation/9EC8C2D8-9AF4-41B8-BA75-246A3E19444A/d/IntelliJ IDEA.app/Contents/lib/idea_rt.jar=60212:/private/var/folders/0j/38__jzzd1x30yf1srqhhqsl40000gn/T/AppTranslocation/9EC8C2D8-9AF4-41B8-BA75-246A3E19444A/d/IntelliJ IDEA.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/tools.jar:/Users/fb/Downloads/bugtestorino/out/test/classes:/Users/fb/Downloads/bugtestorino/libs/karate-core-1.0.0.jar:/Users/fb/Downloads/bugtestorino/libs/karate-apache-1.0.0.jar:/Users/fb/Downloads/bugtestorino/libs/karate-junit4-1.0.0.jar:/Users/fb/.gradle/caches/modules-2/files-2.1/info.picocli/picocli/3.9.2/60911f3b35e76b442d47e764758f3ecc3aa876c2/picocli-3.9.2.jar:/Users/fb/.gradle/caches/modules-2/files-2.1/org.slf4j/jcl-over-slf4j/1.7.25/f8c32b13ff142a513eeb5b6330b1588dcb2c0461/jcl-over-slf4j-1.7.25.jar:/Users/fb/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.25/bccda40ebc8067491b32a88f49615a747d20082d/slf4j-jdk14-1.7.25.jar:/Users/fb/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.25/da76ca59f6a57ee3102f8f9bd9cee742973efa8a/slf4j-api-1.7.25.jar:/private/var/folders/0j/38__jzzd1x30yf1srqhhqsl40000gn/T/AppTranslocation/9EC8C2D8-9AF4-41B8-BA75-246A3E19444A/d/IntelliJ IDEA.app/Contents/lib/junit-4.12.jar:/private/var/folders/0j/38__jzzd1x30yf1srqhhqsl40000gn/T/AppTranslocation/9EC8C2D8-9AF4-41B8-BA75-246A3E19444A/d/IntelliJ IDEA.app/Contents/lib/hamcrest-core-1.3.jar:/private/var/folders/0j/38__jzzd1x30yf1srqhhqsl40000gn/T/AppTranslocation/9EC8C2D8-9AF4-41B8-BA75-246A3E19444A/d/IntelliJ IDEA.app/Contents/plugins/cucumber-java/lib/cucumber-jvm-formatter.jar" cucumber.api.cli.Main --format org.jetbrains.plugins.cucumber.java.run.CucumberJvmSMFormatter --monochrome /Users/fb/Downloads/bugtestorino/src/test/java/test.feature
command: cucumber.api.cli.Main --format org.jetbrains.plugins.cucumber.java.run.CucumberJvmSMFormatter --monochrome /Users/fb/Downloads/bugtestorino/src/test/java/test.feature
Exception in thread "main" picocli.CommandLine$UnmatchedArgumentException: Unknown option: --format
    at picocli.CommandLine$Interpreter.parse(CommandLine.java:7221)
    at picocli.CommandLine$Interpreter.parse(CommandLine.java:7085)
    at picocli.CommandLine.parse(CommandLine.java:824)
    at picocli.CommandLine.populateCommand(CommandLine.java:777)
    at com.intuit.karate.RunnerOptions.parseStringArgs(RunnerOptions.java:85)
    at com.intuit.karate.RunnerOptions.parseCommandLine(RunnerOptions.java:109)
    at com.intuit.karate.IdeUtils.exec(IdeUtils.java:55)
    at cucumber.api.cli.Main.main(Main.java:36)

Process finished with exit code 1
ptrthomas commented 5 years ago

@backfloep not sure really. can you refer this: https://github.com/intuit/karate/wiki/Developer-Guide

and all you need to do is edit the version in your gradle build file to 1.0.0 - you can suggest edits to that wiki

backfloep commented 5 years ago

@ptrthomas Thanks for the hint. Now I run into "Too Many Files Open" Error... Do I have to adjust this system wide?

/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/bin/java -ea -Didea.test.cyclic.buffer.size=1048576 "-javaagent:/private/var/folders/0j/38__jzzd1x30yf1srqhhqsl40000gn/T/AppTranslocation/9EC8C2D8-9AF4-41B8-BA75-246A3E19444A/d/IntelliJ IDEA.app/Contents/lib/idea_rt.jar=62758:/private/var/folders/0j/38__jzzd1x30yf1srqhhqsl40000gn/T/AppTranslocation/9EC8C2D8-9AF4-41B8-BA75-246A3E19444A/d/IntelliJ IDEA.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/private/var/folders/0j/38__jzzd1x30yf1srqhhqsl40000gn/T/AppTranslocation/9EC8C2D8-9AF4-41B8-BA75-246A3E19444A/d/IntelliJ IDEA.app/Contents/lib/idea_rt.jar:/private/var/folders/0j/38__jzzd1x30yf1srqhhqsl40000gn/T/AppTranslocation/9EC8C2D8-9AF4-41B8-BA75-246A3E19444A/d/IntelliJ IDEA.app/Contents/plugins/junit/lib/junit-rt.jar:/private/var/folders/0j/38__jzzd1x30yf1srqhhqsl40000gn/T/AppTranslocation/9EC8C2D8-9AF4-41B8-BA75-246A3E19444A/d/IntelliJ IDEA.app/Contents/plugins/junit/lib/junit5-rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/tools.jar:/Users/fb/Downloads/bugtestorino/out/test/classes:/Users/fb/.m2/repository/com/intuit/karate/karate-apache/1.0.0/karate-apache-1.0.0.jar:/Users/fb/.m2/repository/com/intuit/karate/karate-junit4/1.0.0/karate-junit4-1.0.0.jar:/Users/fb/.m2/repository/com/intuit/karate/karate-core/1.0.0/karate-core-1.0.0.jar:/Users/fb/.m2/repository/org/apache/httpcomponents/httpmime/4.5.5/httpmime-4.5.5.jar:/Users/fb/.m2/repository/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.jar:/Users/fb/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.25/jcl-over-slf4j-1.7.25.jar:/Users/fb/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar:/Users/fb/.m2/repository/org/openjfx/javafx-controls/12-ea+9/javafx-controls-12-ea+9.jar:/Users/fb/.m2/repository/junit/junit/4.12/junit-4.12.jar:/Users/fb/.m2/repository/com/jayway/jsonpath/json-path/2.1.0/json-path-2.1.0.jar:/Users/fb/.m2/repository/info/cukes/cucumber-java/1.2.5/cucumber-java-1.2.5.jar:/Users/fb/.m2/repository/org/yaml/snakeyaml/1.18/snakeyaml-1.18.jar:/Users/fb/.m2/repository/de/siegmar/fastcsv/1.0.3/fastcsv-1.0.3.jar:/Users/fb/.m2/repository/info/picocli/picocli/3.0.1/picocli-3.0.1.jar:/Users/fb/.m2/repository/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.jar:/Users/fb/.m2/repository/commons-codec/commons-codec/1.10/commons-codec-1.10.jar:/Users/fb/.m2/repository/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25.jar:/Users/fb/.m2/repository/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar:/Users/fb/.m2/repository/org/openjfx/javafx-graphics/12-ea+9/javafx-graphics-12-ea+9.jar:/Users/fb/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/Users/fb/.m2/repository/net/minidev/json-smart/2.2/json-smart-2.2.jar:/Users/fb/.m2/repository/info/cukes/cucumber-core/1.2.5/cucumber-core-1.2.5.jar:/Users/fb/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:/Users/fb/.m2/repository/org/openjfx/javafx-base/12-ea+9/javafx-base-12-ea+9.jar:/Users/fb/.m2/repository/net/minidev/accessors-smart/1.1/accessors-smart-1.1.jar:/Users/fb/.m2/repository/org/ow2/asm/asm/5.0.3/asm-5.0.3.jar" com.intellij.rt.execution.junit.JUnitStarter -ideVersion5 -junit4 TestRunner
14:16:19.370 [main] INFO com.intuit.karate - karate.env system property was: null 
14:16:19.481 [main] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $14:16:26.072 [main] INFO com.intuit.karate - karate.env system property was: null 
14:16:26.114 [main] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
java.lang.RuntimeException: java.io.FileNotFoundException: target/surefire-reports/test-1_3820.log (Too many open files)
    at com.intuit.karate.shell.FileLogAppender.<init>(FileLogAppender.java:61)
    at com.intuit.karate.shell.FileLogAppender.<init>(FileLogAppender.java:46)
    at com.intuit.karate.core.ExecutionContext.getLogAppender(ExecutionContext.java:79)
    at com.intuit.karate.core.ScenarioExecutionUnit.<init>(ScenarioExecutionUnit.java:67)
    at com.intuit.karate.core.Feature.getScenarioExecutionUnits(Feature.java:90)
    at com.intuit.karate.core.FeatureExecutionUnit.init(FeatureExecutionUnit.java:60)
    at com.intuit.karate.junit4.FeatureInfo.<init>(FeatureInfo.java:62)
    at com.intuit.karate.junit4.Karate.describeChild(Karate.java:114)
    at com.intuit.karate.junit4.Karate.describeChild(Karate.java:56)
    at org.junit.runners.ParentRunner.getDescription(ParentRunner.java:352)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.io.FileNotFoundException: target/surefire-reports/test-1_3820.log (Too many open files)
    at java.io.RandomAccessFile.open0(Native Method)
    at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
    at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
    at com.intuit.karate.shell.FileLogAppender.<init>(FileLogAppender.java:57)
    ... 14 more

Process finished with exit code 254

This Error occures only if the loaded .csv file (used for dynamic outline) has more than approx. 3300 rows.

ptrthomas commented 5 years ago

@backfloep wow. who would have thought

okay, try this, can you add a @parallel=false tag for the Scenario Outline and try again

backfloep commented 5 years ago

@ptrthomas I already tried. Unfortunately no difference.

ptrthomas commented 5 years ago

@backfloep try now. if that doesn't work, I'll put in a check that stops creating files if the number of scenarios is > 500 ;)

backfloep commented 5 years ago

@ptrthomas The fix didn't help unfortunately. Still the same error. 

/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/bin/java -ea -Didea.test.cyclic.buffer.size=1048576 "-javaagent:/private/var/folders/0j/38__jzzd1x30yf1srqhhqsl40000gn/T/AppTranslocation/9EC8C2D8-9AF4-41B8-BA75-246A3E19444A/d/IntelliJ IDEA.app/Contents/lib/idea_rt.jar=49735:/private/var/folders/0j/38__jzzd1x30yf1srqhhqsl40000gn/T/AppTranslocation/9EC8C2D8-9AF4-41B8-BA75-246A3E19444A/d/IntelliJ IDEA.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/private/var/folders/0j/38__jzzd1x30yf1srqhhqsl40000gn/T/AppTranslocation/9EC8C2D8-9AF4-41B8-BA75-246A3E19444A/d/IntelliJ IDEA.app/Contents/lib/idea_rt.jar:/private/var/folders/0j/38__jzzd1x30yf1srqhhqsl40000gn/T/AppTranslocation/9EC8C2D8-9AF4-41B8-BA75-246A3E19444A/d/IntelliJ IDEA.app/Contents/plugins/junit/lib/junit-rt.jar:/private/var/folders/0j/38__jzzd1x30yf1srqhhqsl40000gn/T/AppTranslocation/9EC8C2D8-9AF4-41B8-BA75-246A3E19444A/d/IntelliJ IDEA.app/Contents/plugins/junit/lib/junit5-rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/lib/tools.jar:/Users/fb/Downloads/bugtestorino/out/test/classes:/Users/fb/.m2/repository/com/intuit/karate/karate-apache/1.0.1/karate-apache-1.0.1.jar:/Users/fb/.m2/repository/com/intuit/karate/karate-junit4/1.0.1/karate-junit4-1.0.1.jar:/Users/fb/.m2/repository/com/intuit/karate/karate-core/1.0.1/karate-core-1.0.1.jar:/Users/fb/.m2/repository/org/apache/httpcomponents/httpmime/4.5.5/httpmime-4.5.5.jar:/Users/fb/.m2/repository/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.jar:/Users/fb/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.25/jcl-over-slf4j-1.7.25.jar:/Users/fb/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar:/Users/fb/.m2/repository/org/openjfx/javafx-controls/12-ea+9/javafx-controls-12-ea+9.jar:/Users/fb/.m2/repository/junit/junit/4.12/junit-4.12.jar:/Users/fb/.m2/repository/com/jayway/jsonpath/json-path/2.1.0/json-path-2.1.0.jar:/Users/fb/.m2/repository/info/cukes/cucumber-java/1.2.5/cucumber-java-1.2.5.jar:/Users/fb/.m2/repository/org/yaml/snakeyaml/1.18/snakeyaml-1.18.jar:/Users/fb/.m2/repository/de/siegmar/fastcsv/1.0.3/fastcsv-1.0.3.jar:/Users/fb/.m2/repository/info/picocli/picocli/3.0.1/picocli-3.0.1.jar:/Users/fb/.m2/repository/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.jar:/Users/fb/.m2/repository/commons-codec/commons-codec/1.10/commons-codec-1.10.jar:/Users/fb/.m2/repository/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25.jar:/Users/fb/.m2/repository/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar:/Users/fb/.m2/repository/org/openjfx/javafx-graphics/12-ea+9/javafx-graphics-12-ea+9.jar:/Users/fb/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/Users/fb/.m2/repository/net/minidev/json-smart/2.2/json-smart-2.2.jar:/Users/fb/.m2/repository/info/cukes/cucumber-core/1.2.5/cucumber-core-1.2.5.jar:/Users/fb/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:/Users/fb/.m2/repository/org/openjfx/javafx-base/12-ea+9/javafx-base-12-ea+9.jar:/Users/fb/.m2/repository/net/minidev/accessors-smart/1.1/accessors-smart-1.1.jar:/Users/fb/.m2/repository/org/ow2/asm/asm/5.0.3/asm-5.0.3.jar" com.intellij.rt.execution.junit.JUnitStarter -ideVersion5 -junit4 TestRunner
15:59:17.650 [main] INFO com.intuit.karate - karate.env system property was: null 
15:59:17.798 [main] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $15:59:23.903 [main] INFO com.intuit.karate - karate.env system property was: null 
15:59:23.933 [main] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
java.lang.RuntimeException: java.io.FileNotFoundException: target/surefire-reports/test-1_3820.log (Too many open files)
    at com.intuit.karate.shell.FileLogAppender.<init>(FileLogAppender.java:62)
    at com.intuit.karate.shell.FileLogAppender.<init>(FileLogAppender.java:47)
    at com.intuit.karate.core.ExecutionContext.getLogAppender(ExecutionContext.java:79)
    at com.intuit.karate.core.ScenarioExecutionUnit.<init>(ScenarioExecutionUnit.java:67)
    at com.intuit.karate.core.Feature.getScenarioExecutionUnits(Feature.java:90)
    at com.intuit.karate.core.FeatureExecutionUnit.init(FeatureExecutionUnit.java:60)
    at com.intuit.karate.junit4.FeatureInfo.<init>(FeatureInfo.java:62)
    at com.intuit.karate.junit4.Karate.describeChild(Karate.java:114)
    at com.intuit.karate.junit4.Karate.describeChild(Karate.java:56)
    at org.junit.runners.ParentRunner.getDescription(ParentRunner.java:352)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.io.FileNotFoundException: target/surefire-reports/test-1_3820.log (Too many open files)
    at java.io.RandomAccessFile.open0(Native Method)
    at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
    at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
    at com.intuit.karate.shell.FileLogAppender.<init>(FileLogAppender.java:58)
    ... 14 more

Process finished with exit code 254
ptrthomas commented 5 years ago

@backfloep ok one more time. FWIW it worked fine on my mac all along with 6300 files, so maybe you should try re-start your system once and try before the git pull

backfloep commented 5 years ago

@ptrthomas After restarting and building the source again, I got rid of the "Too Many Open Files" Error. But unfortunately, now after row 499 of 6300 rows in the csv I get this error:

com.intuit.karate.exception.KarateException: test.feature:15 - null

    at ✽.* print 'asdfg' (test.feature:15)

The test scenario I use for validation purpose is pretty simple:

Feature: StackOverflow Test

  @parallel=false
  Scenario Outline: Produce StackOverflow

    * def contractAction = '<contract>'
    * def mobResAction = '<mobileRessource>'
    * def netProfileAction = '<netProfile>'
    * def dataAccessAction = '<dataAccess>'
    * def voiceMailAction = '<voiceMail>'
    * def mobileServiceAction = '<mobileService>'
    * def result = '<result>'
    * def businessUseCase = "ValMatrixTest"

    * print 'asdfg'

    Examples:
    | read('invalid.csv') |

Do you have any idea what is going wrong?

ptrthomas commented 5 years ago

@backfloep nope. cannot replicate. can you upload another zip ?

backfloep commented 5 years ago

@ptrthomas There you go: bugtest.zip

ptrthomas commented 5 years ago

@backfloep ok can you try now

backfloep commented 5 years ago

@ptrthomas It works like charm! Thank you, Peter!

ptrthomas commented 5 years ago

released 0.9.2

bharatslalchandani commented 5 years ago

@ptrthomas based on above conversation seems we have fixed issue of restricting parallel execution for scenario outline by giving @parallel=false tag above scenario outline (not the scenario) tag in 0.9.2. Is that correct? We are using 0.9.1 and tried @parallel=false above scenario outline (not the scenario) but it is still running in parallel. Thanks in advance. Comment refer from above: https://github.com/intuit/karate/issues/661#issuecomment-458133119

ptrthomas commented 5 years ago

@bharatslalchandani follow this process please, thanks: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue