1.0 release thread

[ptrthomas]

this issue is to provide updates and collect feedback. also will hold some images needed for the wiki

this wiki page will hold details: https://github.com/intuit/karate/wiki/1.0-upgrade-guide

[ptrthomas]

@joelpramos ok, how about naming that as callonce so that it is just the JS version of the keyword

[joelpramos]

@ptrthomas is there a new version of the debugger for VS Code? Was having a lot of issues and I now realized that it's the debugger - seems like the background and the scenario execution don't share the same engine / the engine is set to null when I'm using breakpoints.

My current version in VS Code is 0.9.5.

Super weird but I haven't discounted yet that it could be due to my setup using Spring Boot (see discussion here / my retests for others landing on this comment https://github.com/intuit/karate/issues/751#issuecomment-731326676)

When I trigger the scenarios via the Runner API it works fine. I haven't looked too much into the code for the debugger or did a very deep triage of this.

[ptrthomas]

@joelpramos I hope its because there's a second version of karate in deps maybe because you didn't remove karate-apache 0.9.5 or etc.

paging @ivangsa in case some of the recent changes we worked on broke this - refer #1399

would it be possible to replicate ? cc @kirksl

[joelpramos]

Confirmed I only have karate-junit5. v0.9.5 is the version of the Karate Runner in VS Code but that's the latest in the marketplace. I will try to replicate using standalone JAR version first all will provide some feedback soon.

[ivangsa]

Hi, this is already happening in version 0.9.9.RC1 but I'll have a look into it... I think I have never used this debug console with karate

[joelpramos]

Interesting... our team always used in 0.9.5 and it improves productivity to write scripts a lot (especially in large flows for UI testing) as you can try it out, paste in the code, try new command, paste, etc. Same way you can just print variables to see references to triage issues etc.

There were a couple glitches some times but it's like after going back and forth, hot swap etc. multiple times so nothing that we ever really worried about.

[jkeys089]

I am getting the following error when using the fat jar (either 0.9.9.RC2 or the latest develop branch):

org.graalvm.polyglot.PolyglotException: SyntaxError: No language for id regex found. Supported languages are: [js]

The issue only occurs when I attempt to use a regular expression. E.g. add the following line to karate-config.js

karate.log('a'.replace(/a/g, 'b'))

[joelpramos]

@ptrthomas was doing some testing this morning noticed a commit where the relationship between Step and Feature is removed. Seems like a large commit so I'm sure there was a reason to remove it but steps in the "Background" section don't have a scenario associated with so it creates a nullpointer when trying to get the associated feature via the scenario. I've noticed while using the debugger:

Maybe because of Background steps there's a good use case to keep the Feature associated with the Step, not sure the impacts to the changes being implemented (seems like parsing results into a Feature so are you trying to use results to re-execute tests? kind of curious).

[ptrthomas]

@joelpramos thank you for reporting this, indeed my miss. what's new in the code is being able to serialize a full feature-result into JSON and back again, which now gives us a way to split a test suite across different physical machines and aggregate results. I'll fix this right away and this time put in a comment to make it clear

[ptrthomas]

@joelpramos should be fixed now

[xbranko]

There is an issue with com.intuit.karate.Main it has the following includes:

import ch.qos.logback.classic.Level;
import ch.qos.logback.classic.Logger;

when trying to run I am getting:

java.lang.ClassCastException: class org.apache.logging.slf4j.Log4jLogger cannot be cast to class ch.qos.logback.classic.Logger 
(org.apache.logging.slf4j.Log4jLogger and ch.qos.logback.classic.Logger are in unnamed module of loader 'app')

Just wanted to bring it to your attention. This is observed in com.intuit.karate:karate-core:0.9.9.RC2.

[ptrthomas]

@xbranko works for me. it would help if you can follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue

[xbranko]

Will do. Sorry.

[xbranko]

@ptrthomas you were right. In the simple example all is good. Then I tried to see what is causing the problem, and it turns out it is the slf4j/log4j dependency, in my case brought in by the following line in build.gradle:

implementation 'org.springframework.boot:spring-boot-starter-log4j2'

When running, the following messages appear on the STDERR:

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-slf4j-impl/2.13.3/xx/log4j-slf4j-impl-2.13.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/.gradle/caches/modules-2/files-2.1/ch.qos.logback/logback-classic/1.2.3/yy/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

and the class cast exception is thrown.

If you need/want I can zip the example and share it with you -- just let me know.

I would imagine that many enterprise development projects use slf4j and/or log4j, if not directly, at least transitively, as it is the case for my projects that are all spring based. Would you consider removing logback dependency, or, if not, having two versions of Main, one as it is, and another without logback ? As far as I can tell, the only line that really needs logback is line 205, where the log level is set to WARN.

Thank you for creating such an amazing framework.

[ptrthomas]

@xbranko thanks this is super helpful, just made some changes - if you can use the developer guide to verify in your env that would be great: https://github.com/intuit/karate/wiki/Developer-Guide

[xbranko]

@ptrthomas thanks for the fix! It works now (in the develop branch).

In my project we use gradle. Gradle has a notion of implementation dependencies for code, and testImplementation for test dependencies. I used testImplementation to bring karate-core and karate-junit5 and their transitive dependencies. This is all how it should be.

The problem I ran into was that one of my code implementation dependencies transitively brought in a different version of classgraph (io.github.classgraph:classgraph:4.8.69), which was found before the desired one, brought in by karate (io.github.classgraph:classgraph:4.8.93). Once I explicitly added the code implementation dependency to io.github.classgraph:classgraph:4.8.93 it was all good. Don't really like "polluting" build.gradle file with non-direct dependencies -- not sure how to correctly deal with this, just wanted to mention it.

The exception was due to the missing method acceptPaths:

java.lang.NoSuchMethodError: 'io.github.classgraph.ClassGraph io.github.classgraph.ClassGraph.acceptPaths(java.lang.String[])'
    at com.intuit.karate.resource.ResourceUtils.<clinit>(ResourceUtils.java:93)
    at com.intuit.karate.Runner$Builder.relativeTo(Runner.java:345)
    at com.intuit.karate.junit5.Karate.relativeTo(Karate.java:61)
        ...

[ptrthomas]

@xbranko thanks ! I decided to add classgraph to the list of jars that we maven-shade, I'm guessing that it is quite popular + pervasive, and fortunately it is super-lightweight - so do see if that helps. thanks for the report 👍

[xbranko]

@ptrthomas first off hats off to you! Many thanks for your agility and willingness to help and promptly fixing the issues!

With the latest karate code (develop branch) my tests are now getting the following exception:

java.lang.NoSuchFieldError: packageAcceptReject
    at karate.io.github.classgraph.ClassGraph.acceptPaths(ClassGraph.java:694)
    at com.intuit.karate.resource.ResourceUtils.<clinit>(ResourceUtils.java:93)
    at com.intuit.karate.Runner$Builder.relativeTo(Runner.java:345)
    at com.intuit.karate.junit5.Karate.relativeTo(Karate.java:61)
        ...

Would it be possible to add sources for karate jars in pre-release maven phase, in addition to release phase? That would be of great help to us, so we can better help you!

[ptrthomas]

@xbranko that is weird, in this case I guess the libs you have in your project means a maven exclude or explicit version force is the only option. I don't know how to add sources - I think it will work only if I do a maven release. if anyone here has any inputs or better ideas, let me know

[ptrthomas]

@xbranko turns out there was a solution for the error you faced. can you try again

[joelpramos]

Not sure how it worked before (something that Nashorn did in a smart way?) but using stringVariable.bytes no longer works but stringVariable.getBytes() works.

So the following example in the Karate doc:

function fn(creds) {
  var temp = creds.username + ':' + creds.password;
  var Base64 = Java.type('java.util.Base64');
  var encoded = Base64.getEncoder().encodeToString(temp.bytes);
  return 'Basic ' + encoded;
}

Should change to:

function fn(creds) {
  var temp = creds.username + ':' + creds.password;
  var Base64 = Java.type('java.util.Base64');
  var encoded = Base64.getEncoder().encodeToString(temp.getBytes());
  return 'Basic ' + encoded;
}

Maybe Nashorn had some code to find getters in Java to make it more "javascript like".

[ptrthomas]

@joelpramos yes I'd noted this as well: https://github.com/intuit/karate/wiki/1.0-upgrade-guide#javabean-property-short-cuts-dont-work-in-all-cases

what I think is happening is within the round brackets, none of the java-bean magic can happen.

BTW I'm going offline, but I had a late thought that instead of expecting back-ticks for that PR, it may be enough to check for the ${} pattern or / even just add the back-ticks from the code. do think it over thx.

[joelpramos]

Good, missed that from the guide. Yeah can add the backticks automatically if the ${anything} exists, makes it smoother

[joelpramos]

@joelpramos thank you for reporting this, indeed my miss. what's new in the code is being able to serialize a full feature-result into JSON and back again, which now gives us a way to split a test suite across different physical machines and aggregate results. I'll fix this right away and this time put in a comment to make it clear

Still an issue when I forced an error in the karate-config.js - the debug info tries to write the error based on the first line of the background which fails because of the nullpointer on the scenario. Should be this.getFeature() . I'm also going offline but can fix it in my PR tomorrow or during the week when I made that tweak for the backticks.

[ptrthomas]

@joelpramos thanks ! fixed now

[xbranko]

@xbranko turns out there was a solution for the error you faced. can you try again

Confirming all is as expected and working fine in the latest develop branch. Many thanks for fixing this!

[xbranko]

I don't know how to add sources - I think it will work only if I do a maven release. if anyone here has any inputs or better ideas, let me know

Trying to answer my own question, I just tried, If the following plugin directive is added to the build block (lines 60-82 in the karate-parent/pom.xml file), the *-sources.jar are generated and appear in the local maven repository:

                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-source-plugin</artifactId>
                        <version>2.2.1</version>
                        <executions>
                            <execution>
                                <id>attach-sources</id>
                                <goals>
                                    <goal>jar-no-fork</goal>
                                </goals>
                            </execution>
                        </executions>
                    </plugin>

This is copy of the lines 89-101 in the karate-parent/pom.xml file. If this is acceptable, perhaps those lines can be moved there, to avoid duplication and confusion.

[ptrthomas]

@xbranko okay, see how it works now, thx

[xbranko]

@xbranko okay, see how it works now, thx

Confirming all is good. Source jars getting created in local maven repository. Thanks for adding this!

[joelpramos]

In the ScenarioFileReader class when reading a file that has the prefix "file:" it seems like it's using the relative path where Karate is executing. Was it intentional? Seems like the ability of using absolute paths is gone with line 125 of ResourceUtils (2nd screenshot).

[xbranko]

I've just tried running standalone karate-core jar from my local maven cache, and it is failing.

java -jar <path_to_jar>/karate-core-2.0.0.jar -h

produces:

Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
        at com.intuit.karate.FileUtils.<clinit>(FileUtils.java:48)
        at com.intuit.karate.Main.main(Main.java:220)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        ... 2 more

Few more jars need to be added to it. slf4j-api-xx.yy.zz.jar that contains org/slf4j/LoggerFactory and most likely some other jars that get pulled transitively when running in an IDE, where all is good. Perhaps a smoke test or two could be added to run in the post install phase to verify that the newly created jar is OK.

[ptrthomas]

@xbranko I think that is fine. use the "fatjar" and you won't have this problem. the fatjar is now around 50 MB. see the dev guide for build instructions: https://github.com/intuit/karate/wiki/Developer-Guide#build-standalone-karate-robot-jar

[ptrthomas]

@joelpramos the workingDir is only used for formatting the path, so if you gave file:/foo it should use /foo as expected. I think it is fine, see if you can write a test-case to fail it

[joelpramos]

@joelpramos the workingDir is only used for formatting the path, so if you gave file:/foo it should use /foo as expected. I think it is fine, see if you can write a test-case to fail it

Yeah ignore me... I probably didn't save my file or something when I was originally testing that change and was using the path without the 'file:'. The behavior is correct.

[ivangsa]

In previous version 0.9.6 karate was resilient ignoring params set with a null value, in latest 0.9.9.RC2 this throws an error

We would like to keep relying on previous behavior as it feels more resilient.

The reason is we generate parametrizable features from OpenAPI definitions, that then are called from different tests with different parameters... Relying on this, caller can forget about setting non-required parameters.

This illustrates an simple scenario that works on 0.9.6 and fails on 0.9.9.RC2

Scenario:
* url 'https://jsonplaceholder.typicode.com/todos'
* def arg = { params: { someparam: 'value'}}
Given path '', 1
And param nullparam = arg.params.nullparam
And param someparam = arg.params.someparam
And param arrayparam = [#(arg.params.someparam), #(arg.params.nullparam)]
When method get
Then status 200
* match response contains { id: '#number', title: '#string' }

I have a PR for this.

[joelpramos]

@ptrthomas before I go down a rabbit hole let me know if this is a limitation of the Graal engine you mention in the upgrade guide.

When calling a feature in the background (and not assigning a it to a variable) that reads a .json file with keys, those json key variables are not accessible in the caller context nor in any features called from there (obviously if not accessible in the caller already).

js-read.feature

js-read-reuse.feature

js-read-3.js

js-read-called.feature

[joelpramos]

Something I noticed is that a Feature file with only Background section will not have those background steps executed. The background steps are included with each ScenarioRuntime and since there's no Scenario parsed and so the background steps are not executed. Might be worth a note in the upgrade guide - I see I had a few features just with Background sections mostly to reuse and init parameters.

After putting those steps into a Scenario it still didn't work so my gut feeling is because of that limitation of Graal.

[ptrthomas]

@joelpramos can you check if karate.set('varname', value) suffices as a workaround, in that case I'll leave it as is. it may actually be a good thing, local hiding of variables. for e.g. this particular diff: https://github.com/intuit/karate/wiki/1.0-upgrade-guide#json-variables-no-longer-mutable-by-called-features

[joelpramos]

Yeah agree could be a good thing. Seems like we if assign the result of the called feature to a variable (see below assigned to varInContext) that we have access to the variables within that feature.

Feature:

Background:

Scenario:
    * call read 'js-read-3.json'
    * def storedVar = call read 'js-read-3.json'
    #* karate.set('storedVar', storedVar)

Feature:

Background:
    * def varInContext = callonce read('this:js-read-reuse.feature')
    * call read('this:js-read-reuse.feature')

Scenario: calling feature that will read a json file and then call a feature that uses data from that json
    * match varInContext.storedVar.thirderror[0].id == 1
    * match varInContext.thirderror[0].id == 1
    * match storedVar.thirderror[0].id == 1
    * match thirderror[0].id == 1
    * def result = call read('js-read-called.feature@name=checkReadingThirdJsonKeys')

3rd matching condition (accessing storedVar) fails. Trying to use the Karate API directly produces the same result ( * karate.call(true, 'this:js-read-reuse.feature') ).

I'll give it some thought as on one hand I agree hiding variables could be good but maybe the Karate API could force that shared scope concept that existed before.

[joelpramos]

@ptrthomas the above works and it's how we use call read to call the reusable feature.

Will make scope shared: * call read('this:js-read-reuse.feature')

Will not make scope shared: * call read 'this:js-read-reuse.feature'

I think you have some documentation on this somewhere already, it rings as the space and correlation with shared scope rings a bell.

There's still some issue in my own tests that I can't seem to reproduce with a unit test but it must related to this. I'll check in these unit tests regardless cause I ended up doing a bunch to try to reproduce. Still trying to find some breaking issues and this one regardless doesn't seem to be worth my time to investigate (probably something minor that can easily be refactored).

[ptrthomas]

@joelpramos see last row of table: https://github.com/intuit/karate#call-vs-read

[Lorkenpeist]

replace does not seem to work correctly anymore. After replacing a string, the toString() method still returns the raw string. Consider this test:

Scenario:
  * def text = "words that need to be {replaced}"
  * replace text.{replaced} = "correct"
  * match text == "words that need to be correct"
  * match text.toString() == "words that need to be correct"

Both matches should be equivalent, and in 0.9.6 they both work correctly. But in 0.9.9.RC2 the second match fails with the following error message:

$ | not equal (STRING:STRING)
  'words that need to be {replaced}'
  'words that need to be correct'

[ptrthomas]

@Lorkenpeist excellent catch thank you ! fixed in develop. we should have an RC3 in a day or 2 but you can use the developer guide (see wiki) to verify.

[ptrthomas]

all: 0.9.9.RC3 has been released: https://twitter.com/KarateDSL/status/1349603552478461953

[Lorkenpeist]

@Lorkenpeist excellent catch thank you ! fixed in develop. we should have an RC3 in a day or 2 but you can use the developer guide (see wiki) to verify.

Looks good in 0.9.9.RC3, thanks!

[joelpramos]

Finally figured out the issue @ptrthomas - it's not with the read. When using a dynamic variable in the Examples table we loose access to any variable or context defined in the background. There's probably some re-init of the engine somewhere?

Below scenario no 1 works but scenario no 2 fails with the following:

20:11:48.722 [main] ERROR com.intuit.karate - classpath:com/intuit/karate/core/js-read-2.feature:20
* match data.error[__num].id == __row.id
>>>> js failed:
01: data
<<<<
org.graalvm.polyglot.PolyglotException: ReferenceError: "data" is not defined
- <js>.:program(Unnamed:1)

Feature:

Background:
    * def data = call read 'js-read.json'
    * def test_data = data.error

Scenario Outline: using a scenario outline, try to access background
    * match data.error[0].id == value
    * match test_data == "#present"

Examples:
    | key! | value! |
    | 0   | 1     |
    #| 1   | 2     |

Scenario Outline: using a scenario outline with dynamic variable as data, try to access background variables
    * print __row
    * match data.error[__num].id == __row.id
    * match error[0].id == 1

Examples:
    | test_data |

While doing it I also noticed something that I think is another error but I can't recall whether I used it before:

Feature:

Background:
    * def data = call read 'js-read.json'
    * def test_data = data.error

Scenario Outline: using a scenario outline, try to access background
    * match data.error[key].id == value
    * match test_data == "#present"

Examples:
    | key! | value! |
    | 0   | 1     |
    | 1   | 2     |

Following error:

20:14:21.516 [main] ERROR com.intuit.karate - classpath:com/intuit/karate/core/js-read-2.feature:8
* match data.error[key].id == value
Could not parse token starting at position 7. Expected ?, ', 0-9, * 

It triggers when I try to use a variable as the array index - data.error[key].id == value . Seems weird as position 7 is the match and so it might be some regex that is broken.

[joelpramos]

I'm logging off now so won't be picking it up immediately. I also noticed some weird behaviors with the driver.input() when we pass something like this:

`

• def credentials = """ { "username" : '#(user)', "password" : '#(pass)' } ... some code here, call a reusable feature with the following

• input(loginScreen.username, credentials.username) """ `

In which the user variable is null (in my case was due to background issue reported above) I get me the following error:

*** step failed: >>>> js failed:
01: input(loginScreen.username, credentials.username)
<<<<
org.graalvm.polyglot.PolyglotException: TypeError: invokeMember (input) on com.intuit.karate.driver.chrome.Chrome@5aadeb88 failed due to: Multiple applicable overloads found for method name input (candidates: [Method[public com.intuit.karate.driver.Element com.intuit.karate.driver.DevToolsDriver.input(java.lang.String,java.lang.String)], Method[public default com.intuit.karate.driver.Element com.intuit.karate.driver.Driver.input(java.lang.String,java.lang.String[])]], arguments: [input[name='Login-LoginScreen-LoginDV-username'] (String), DynamicObject<undefined>@2c72c81e (Nullish)])
- <js>.:anonymous(Unnamed:1)

Super edge and only happens to me because of the previous issue but it's cause the second value is null. Might be something worth trying to fail more gracefully if possible.

[ptrthomas]

@joelpramos as I linked earlier, please note that call read 'js-read.json' may not be what you intended.

for the second problem please see: https://github.com/intuit/karate/issues/1280

can you extend my simple example below to replicate:

Feature:

Background:
* def data = [{ name: 'one' }, { name: 'two' }]

Scenario Outline:
* print name

Examples:
| data |

[joelpramos]

Context to "anotherVariable" is lost but context of "data" is accessible.

Feature:

Background:
    * def anotherVariable = 'hello'
    * def data = [{ name: 'one' }, { name: 'two' }]
Scenario Outline:
    * print name
    * print '> ' + anotherVariable

Examples:
    | data |

14:52:14.135 [main] ERROR com.intuit.karate - classpath:com/intuit/karate/core/js-read-2.feature:14
* print '> ' + anotherVariable
>>>> js failed:
01: karate.log('[print]','> ' + anotherVariable)
<<<<
org.graalvm.polyglot.PolyglotException: ReferenceError: "anotherVariable" is not defined
- <js>.:program(Unnamed:1)

[joelpramos]

Context to "anotherVariable" is lost but context of "data" is accessible.

Feature:

Background:
    * def anotherVariable = 'hello'
    * def data = [{ name: 'one' }, { name: 'two' }]
Scenario Outline:
    * print name
    * print '> ' + anotherVariable

Examples:
    | data |

14:52:14.135 [main] ERROR com.intuit.karate - classpath:com/intuit/karate/core/js-read-2.feature:14
* print '> ' + anotherVariable
>>>> js failed:
01: karate.log('[print]','> ' + anotherVariable)
<<<<
org.graalvm.polyglot.PolyglotException: ReferenceError: "anotherVariable" is not defined
- <js>.:program(Unnamed:1)

This line in ScenarioRuntime.java solves it but I'm not sure if it's the best place as I find it odd in only happens when the expression in the Examples table is dynamic. If you think this is good enough I can open the PR.

Simplified unit test (first one fails with current code base, both pass with the change above):

Feature:

Background:
    * def anotherVariable = 'hello'
    * def data = [{ name: 'one' }, { name: 'two' }]

Scenario Outline:
    * match name == "#present"
    * match anotherVariable == "hello"

Examples:
    | data |

Scenario Outline:
    * match name == "#present"
    * match anotherVariable == "hello"

Examples:
    | name |
    | test |

I ended up writing a bunch of unit tests for combinations of reading jsons with space and without space etc so I'll check them all in once we sort this one out.