martinpiper / BDD6502

Behaviour Driven Development with 6502 code
27 stars 2 forks source link

Can cukesplus be made public? And problems running the project #14

Closed markjfisher closed 1 year ago

markjfisher commented 1 year ago

Hi, I've come across your project and decided to use it for some work I'm doing with the FujiNet (https://github.com/FujiNetWIFI/) project I'm involved in.

I'm predominantly a Kotlin user, and tried writing a project that uses your release jar, but uses Kotlin's class extensions to write new Given clauses, e.g.

import com.replicanet.cukesplus.Main
import TestGlue.Glue
import cucumber.api.java.en.Given
import java.io.FileInputStream

fun main(args: Array<String>) {
    val mainArgs = "--monochrome --plugin pretty --plugin html:target/cucumber --plugin json:target/report1.json --glue TestGlue features".split(" ").toTypedArray()
    Main.main(mainArgs)
}

@Throws(Throwable::class)
@Given("^I load xex \"([^\"]*)\"$")
fun Glue.i_load_xex(file: String) {
    val inFile = FileInputStream(file)
    // ... etc
}

This does bring up the listener, but sadly my Glue extensions are not getting recognised in the JVM that is launched. I tried using java.lang.String instead of kotlin's builtin String, but still no joy.

So I thought I'd work at making a Kotlin version that does allow this, based on your codebase (reimplementing in pure Kotlin instead of java), but stripping back most of the C64 code, as I'm predominantly going to be using Atari/MADS/Altirra instead of C64/ACME/VICE, and providing the core functionality but agnostic of target hardware.

My idea is to get an initial setup going, and maybe build it in a way that different platforms could extend the functionality for themselves, as the FujiNet project works on many types of boards. Thus provide 'runners' for each board, but allow people to write Given clauses outside the the BDD framework, which is where I'm currently struggling.

Does this sound like an approach you could see working? I've only really scratched at the codebase's surface. Or maybe there's an easier way for me to write new Given clauses from a separate project I haven't considered? (As that's my main usecase, and porting to Kotlin is a huge direction change for what I'm currently working on).

However, I tried to start, but the first problem I hit is I can't download the source mentioned in pom.xml for https://github.com/martinpiper/CukesPlus.git as it's a private repo of yours.

So rather than rip it out the jar file, I wondered if I could start a dialogue with yourself, and see where it leads :)

Thanks, Mark

martinpiper commented 1 year ago

The command line option "--glue TestGlue" tells Cucumber to find all the @Given/@When/@Then annotated class methods in the "package TestGlue".

As long as, from the JVM's point of view, your method is "public void", belongs to a public class that is in the TestGlue package, and is annotated with @Given/@When/@Then Cucumber should be able to find it.

You might want to try a class/method style like this: https://cucumber.io/docs/guides/10-minute-tutorial/?lang=kotlin

New StepDefs.kt file...

package TestGlue

import io.cucumber.java.PendingException
import io.cucumber.java.en.Given
import org.junit.Assert.*

class StepDefs {
    @Given("today is Sunday")
    @Throws(Exception::class)
    fun today_is_Sunday() {
        // Write code here that turns the phrase above into concrete actions
        throw PendingException()
    }
    ...

When the framework runs, look in target/syntax.html as this will tell you if the new step definition is being found by Cucucmber.

markjfisher commented 1 year ago

Thanks for the quick reply. I hadn't considered what package I was writing these in, I'll try that out now.

My example was a quick hack to see if I could get it working, I don't normally like to write anything outside a class, so the example you've shown would be how I go anyway.

It would also be really nice if the project were available as a standard dependency, instead of having to download it (which adds a huge jar to the git repo). I'm having issues with attaching src etc for general usage, and my idea of rewriting in kotlin was out of need to extend it, but if the above works, I can leave it as is, except for having to download versions manually and trying to sort out src attachments.

Off to try the above out...

markjfisher commented 1 year ago

Ok, having it in the TestGlue package has got me past the error about missing my Given clause, so I'm making progress, I can see in the output:

    And I run the command line: mads -o:test.xex -l:test.lbl test.a         # Glue.i_run_the_command_line(String)
    And I load xex "test.obj"                                               # StepDefs.i_load_xex(String)

I think I now have some add-opens issues:

Given I have a simple 6502 system                                       # Glue.i_have_a_simple_6502_system()
      java.lang.ExceptionInInitializerError
     Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.util.Comparator java.util.TreeMap.comparator accessible: module java.base does not "opens java.util" to unnamed module @2b742ace

I'm using an intellij run config with following VM parameters I thought would work:

-Dcom.replicanet.cukesplus.server.featureEditor -Dcom.replicanet.ACEServer.debug.requests= --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang.reflect=ALL-UNNAMED --add-opens java.base/java.text=ALL-UNNAMED --add-opens java.desktop/java.awt.font=ALL-UNNAMED
martinpiper commented 1 year ago

Yeah I add: --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang.reflect=ALL-UNNAMED --add-opens java.base/java.text=ALL-UNNAMED --add-opens java.desktop/java.awt.font=ALL-UNNAMED

This should work: https://stackoverflow.com/questions/70756414/java-lang-reflect-inaccessibleobjectexception-unable-to-make-field-private-fina

Are you using at least java 1.8?

markjfisher commented 1 year ago

yes, java 17 in fact. I did have a complex setup via WSL2, just removing that from the equation.

markjfisher commented 1 year ago

I tried for a few hours to make some progress, but I'm still getting the same errors, although I was slightly out in the previous errors, it actually starts with a selenium error about not being able to take a screenshot

I'm attempting to run a simple test:

Feature:  Test Feature

  This is a test

  Scenario: Simple test
    Then assert that "1" is true

The server I start with:

fun main(args: Array<String>) {
    val mainArgs = "--monochrome --plugin pretty --plugin html:target/cucumber --plugin json:target/report1.json --glue TestGlue features".split(" ").toTypedArray()
    Main.main(mainArgs)
    // TestRunner.main(emptyArray()) // this also does similar to above
}

I can open the cukes page in browser, but running the simple scenario gives me same error as previously reported:

Current path: <redacted>
The args =[java, -Djava.runtime.name=OpenJDK Runtime Environment, -Djava.vm.version=17.0.7+7-LTS, -Dsun.boot.library.path=...<redacted>, -cp, ...\annotations-13.0.jar, com.replicanet.cukesplus.Main, --monochrome, --plugin, pretty, --plugin, html:target/cucumber, --plugin, json:target/report1.json, --glue, TestGlue, features/test.feature]
FeatureServer starting run...
CukesPlus starting : (C) 2015 Replica Software : www.replicasoftware.com
Default properties file was not loaded: CukesPlus.properties
CukesPlus.properties (The system cannot find the file specified)
Info: Processing features/test.feature
Info: Feature macro processing 'features/test.feature' depth 1 new lines 0 total lines 0
Feature: Test Feature

  This is a test
Failure in after hook:SeleniumGlue.afterHook(Scenario)
Message: java.lang.NullPointerException: Cannot invoke "org.openqa.selenium.TakesScreenshot.getScreenshotAs(org.openqa.selenium.OutputType)" because "ts" is null
    at TestGlue.SeleniumGlue.screenshotToReport(SeleniumGlue.java:185)
    at TestGlue.SeleniumGlue.afterHook(SeleniumGlue.java:37)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        ...
    at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:165)
    at cucumber.runtime.Runtime.run(Runtime.java:121)
    at com.replicanet.cukesplus.Main.run(Main.java:37)
    at com.replicanet.cukesplus.Main.main(Main.java:25)

      java.lang.NullPointerException: Cannot invoke "org.openqa.selenium.TakesScreenshot.getScreenshotAs(org.openqa.selenium.OutputType)" because "ts" is null
        at TestGlue.SeleniumGlue.screenshotToReport(SeleniumGlue.java:185)
        at TestGlue.SeleniumGlue.afterHook(SeleniumGlue.java:37)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        // ...
        at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:165)
        at cucumber.runtime.Runtime.run(Runtime.java:121)
        at com.replicanet.cukesplus.Main.run(Main.java:37)
        at com.replicanet.cukesplus.Main.main(Main.java:25)

  Scenario: Simple test          # features/test.feature:5
    Then assert that "1" is true # Glue.assert_that_true(String)
      java.lang.ExceptionInInitializerError
        at cucumber.deps.com.thoughtworks.xstream.XStream.setupConverters(XStream.java:820)
        at cucumber.deps.com.thoughtworks.xstream.XStream.<init>(XStream.java:574)
        // ...
        at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:165)
        at cucumber.runtime.Runtime.run(Runtime.java:121)
        at com.replicanet.cukesplus.Main.run(Main.java:37)
        at com.replicanet.cukesplus.Main.main(Main.java:25)
        at ✽.Then assert that "1" is true(features/test.feature:6)
      Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.util.Comparator java.util.TreeMap.comparator accessible: module java.base does not "opens java.util" to unnamed module @65b3120a
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
        // ...
        at cucumber.runtime.model.CucumberScenario.run(CucumberScenario.java:44)
        at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:165)
        at cucumber.runtime.Runtime.run(Runtime.java:121)
        at com.replicanet.cukesplus.Main.run(Main.java:37)
        at com.replicanet.cukesplus.Main.main(Main.java:25)

Failed scenarios:
features/test.feature:5 # Scenario: Simple test

1 Scenarios (1 failed)
1 Steps (1 failed)
0m0.082s

java.lang.ExceptionInInitializerError
    at cucumber.deps.com.thoughtworks.xstream.XStream.setupConverters(XStream.java:820)
        // ...
    at com.replicanet.cukesplus.Main.run(Main.java:37)
    at com.replicanet.cukesplus.Main.main(Main.java:25)
    at ✽.Then assert that "1" is true(features/test.feature:6)
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.util.Comparator java.util.TreeMap.comparator accessible: module java.base does not "opens java.util" to unnamed module @65b3120a
    at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
    at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
    at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
    at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
    at cucumber.deps.com.thoughtworks.xstream.core.util.Fields.locate(Fields.java:39)
        // ...
    at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:165)
    at cucumber.runtime.Runtime.run(Runtime.java:121)
    at com.replicanet.cukesplus.Main.run(Main.java:37)
    at com.replicanet.cukesplus.Main.main(Main.java:25)

java.lang.NullPointerException: Cannot invoke "org.openqa.selenium.TakesScreenshot.getScreenshotAs(org.openqa.selenium.OutputType)" because "ts" is null
    at TestGlue.SeleniumGlue.screenshotToReport(SeleniumGlue.java:185)
    at TestGlue.SeleniumGlue.afterHook(SeleniumGlue.java:37)
        // ...
    at cucumber.runtime.Runtime.runAfterHooks(Runtime.java:205)
    at cucumber.runtime.model.CucumberScenario.run(CucumberScenario.java:46)
    at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:165)
    at cucumber.runtime.Runtime.run(Runtime.java:121)
    at com.replicanet.cukesplus.Main.run(Main.java:37)
    at com.replicanet.cukesplus.Main.main(Main.java:25)

FeatureServer run exitstatus = 1

Sorry for the spam, but I'm really struggling to get this off the ground. It seems to want to tie into Selenium to take screenshots.

I did manage to get this working:

@RunWith(CucumberPlus::class)
@CucumberOptions(tags = ["~@ignore"], monochrome = true, glue = ["TestGlue"], format = ["pretty", "html:target/cucumber"], features = ["features"])
class Main {}

This at least runs my feature correctly and passes without any of the selenium errors.

Coming back to part of the reason for this ticket, I'd really like to be able to use your code without just the jar, but I can't compile the BDD6502 project either, as I don't have access to https://github.com/martinpiper/CukesPlus.git:

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.105 s
[INFO] Finished at: 2023-07-10T23:54:12+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project BDD6502: Could not resolve dependencies for project BDD6502:BDD6502:jar:1.0.9-SNAPSHOT: The following artifacts could not be resolved: com.replicanet:cukesplus-core:jar:0.0.2-SNAPSHOT (absent): Could not find artifact com.replicanet:cukesplus-core:jar:0.0.2-SNAPSHOT -> [Help 1]
martinpiper commented 1 year ago

Give me a short while, I think the Selenium error is a problem in my code... What changes did you need to get it running without the access errors?

martinpiper commented 1 year ago

Should be fixed in: https://github.com/martinpiper/BDD6502/commit/76de59bd605a4cfd58680676fe6bdf14e59d673d I do plan to open source CukesPlus but not immediately soon as there are some novel ideas in there...

markjfisher commented 1 year ago

What changes did you need to get it running without the access errors?

Non yet. if you look in that stack trace it eventually still says:

Unable to make field private final java.util.Comparator java.util.TreeMap.comparator accessible: module java.base does not "opens java.util" to unnamed module @65b3120a

I tried jdk 8, which doesn't work, your jar was built with 11 so doesn't load older classes, but ran out of time it being 1am investigating further with jdk 11.

Should be fixed in: 76de59b

I haven't picked this up yet, but I have found that JDK 11 does not throw the stack trace above, but JDK 17 does. I've just switched back and forth between JDK 11 and 17 and it's completely reproducible. 11 tests run no errors, 17 throws all the exceptions mentioned above.

I do get an warning in JDK 11 saying:

Warning: Nashorn engine is planned to be removed from a future JDK release

which may be of some interest. I read that was removed in JDK 15, but don't know what role it plays in the setup yet. There is an openJDK Nashorn according to https://stackoverflow.com/a/69865070/690139

Trying your changes now on top to see what happens

markjfisher commented 1 year ago

Using the latest release now removes the selenium error in JDK17, but the rest of the errors are still there, which is what I was expecting. All fine on JDK 11.

C:\msys64\home\markj\.sdkman\candidates\java\17.0.7-zulu\bin\java.exe ... MainKt
http://127.0.0.1:8001/ace-builds-master/demo/autocompletion.html
Current path: ...
The args =[java, ..., com.replicanet.cukesplus.Main, --monochrome, --plugin, pretty, --plugin, html:target/cucumber, --plugin, json:target/report1.json, --glue, TestGlue, features/test.feature]
FeatureServer starting run...
CukesPlus starting : (C) 2015 Replica Software : www.replicasoftware.com
CukesPlus.properties (The system cannot find the file specified)
Info: Processing features/test.feature
Info: Feature macro processing 'features/test.feature' depth 1 new lines 0 total lines 0
Feature: Test Feature

  This is a test

  Scenario: Simple test          # features/test.feature:5
    Then assert that "1" is true # Glue.assert_that_true(String)
      java.lang.ExceptionInInitializerError
        at cucumber.deps.com.thoughtworks.xstream.XStream.setupConverters(XStream.java:820)
        at cucumber.deps.com.thoughtworks.xstream.XStream.<init>(XStream.java:574)
        ...
        at com.replicanet.cukesplus.Main.run(Main.java:37)
        at com.replicanet.cukesplus.Main.main(Main.java:25)
        at ✽.Then assert that "1" is true(features/test.feature:6)
      Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.util.Comparator java.util.TreeMap.comparator accessible: module java.base does not "opens java.util" to unnamed module @65b3120a
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
        ...
        at cucumber.runtime.model.StepContainer.runSteps(StepContainer.java:39)
        at cucumber.runtime.model.CucumberScenario.run(CucumberScenario.java:44)
        at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:165)
        at cucumber.runtime.Runtime.run(Runtime.java:121)
        at com.replicanet.cukesplus.Main.run(Main.java:37)
        at com.replicanet.cukesplus.Main.main(Main.java:25)

Failed scenarios:
features/test.feature:5 # Scenario: Simple test

1 Scenarios (1 failed)
1 Steps (1 failed)
0m0.087s

java.lang.ExceptionInInitializerError
    at cucumber.deps.com.thoughtworks.xstream.XStream.setupConverters(XStream.java:820)
    at cucumber.deps.com.thoughtworks.xstream.XStream.<init>(XStream.java:574)
    at cucumber.deps.com.thoughtworks.xstream.XStream.<init>(XStream.java:530)
    ...
    at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:165)
    at cucumber.runtime.Runtime.run(Runtime.java:121)
    at com.replicanet.cukesplus.Main.run(Main.java:37)
    at com.replicanet.cukesplus.Main.main(Main.java:25)
    at ✽.Then assert that "1" is true(features/test.feature:6)
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.util.Comparator java.util.TreeMap.comparator accessible: module java.base does not "opens java.util" to unnamed module @65b3120a
    at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
    at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
    ...
    at cucumber.runtime.model.CucumberScenario.run(CucumberScenario.java:44)
    at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:165)
    at cucumber.runtime.Runtime.run(Runtime.java:121)
    at com.replicanet.cukesplus.Main.run(Main.java:37)
    at com.replicanet.cukesplus.Main.main(Main.java:25)

FeatureServer run exitstatus = 1
markjfisher commented 1 year ago

I do plan to open source CukesPlus but not immediately soon as there are some novel ideas in there...

ok, but given you're releasing it in the jar file anyway, the bytecode is decompiled by IntelliJ, so all I'm really missing is the documentation 😄 But if you have some IP in there, fair enough, it's just your docs say:

Setup: Build the jar, ...

which currently is impossible without some major hacking by copying the CukesPlus classes out of the release and adding them as a dependency somehow back into maven, at which point it's just a bit too much effort.

I am however pleased I've found the root cause and can try out bigger features than testing "1" is true 😁

martinpiper commented 1 year ago

My current javas for reference:

java -version openjdk version "1.8.0_302" OpenJDK Runtime Environment (Temurin)(build 1.8.0_302-b08) OpenJDK 64-Bit Server VM (Temurin)(build 25.302-b08, mixed mode)

where java C:\Program Files\Eclipse Foundation\jdk-8.0.302.8-hotspot\bin\java.exe C:\Users\blah.jdks\corretto-1.8.0_252\bin\java.exe C:\Program Files (x86)\Common Files\Oracle\Java\javapath\java.exe

martinpiper commented 1 year ago

I think I am going to have to update Cucumber...

markjfisher commented 1 year ago

Interesting, I must have had a local issue with the JDK8 then, I was getting some errors about source code having been built at 11 (by the byte code version), which I didn't have installed at the time, so assumed it was the downloaded JAR.

markjfisher commented 1 year ago

I'm using sdkman in a MSYS2 environment, so I can install from dozens of versions of java and switch between them on the command line. In IntelliJ, I just add the sdk pointing to the install of the candidates:

image
markjfisher commented 1 year ago

I'm also getting issues linking the steps between features and IntelliJ's view.

image

Several suggestions mention having to add the dependency and its sources https://stackoverflow.com/a/55395238/690139

I'll try and research this further unless you know how to link steps from an external jar.

martinpiper commented 1 year ago

IntelliJ often fails to index and give code hints for syntax in jars. I've not yet found a reliable way to fix it. It's one of the reasons I created the web based editor.

markjfisher commented 1 year ago

Found this open issue, I've added my voice behind : https://youtrack.jetbrains.com/issue/IDEA-157652

markjfisher commented 1 year ago

It's one of the reasons I created the web based editor

Wow, I hadn't appreciated the web client had that functionality. That's really cool.

markjfisher commented 1 year ago

I'm getting so much further with this now, It's compiling my src using chosen assembler. How do I configure directories for outputs etc?

I'm basing my feature on https://github.com/martinpiper/BDD6502/blob/master/features/MachineState.feature The test.a file and its compiled version and labels are all going into the root directory of the project, which is a bit nasty.

Is there a way to specify a default dir for working in? I'd like to configure it to go to something like "build/tests" or similar.

Also, the step to load the exe is failing as it can't find the generated file, again linked to no working directory structure, the step:

And I load xex "test.obj"

sends my step function the name "test.obj" without any kind of path before it. How can I ensure it finds the generated files? I couldn't see you doing anything with paths etc in the example, so wondering how you manage?

Edit: Actually, I see you've added ignores for the generated files in git, which I'm guessing is how you're doing it by letting them be in root.

martinpiper commented 1 year ago

This video shows the web editor being used with macro extension syntax: https://youtu.be/fdeVc2q6oB0?t=118

martinpiper commented 1 year ago

Output directories depend on the assembler and its command line parameters. For example this line: https://github.com/martinpiper/BDD6502/blob/master/features/assemble.feature#L21C95-L21C95

It uses "-o test.prg", the "-o" is for output, the "test.prg" is just a file path. It could just as easily be "target\test.prg" for the output.

The step "I run the command line:" is always relative from the current execution direction.

markjfisher commented 1 year ago

Ah yes, thanks. I was coming to that conclusion. Just wondered if there was some already built way of setting work dirs etc before I put everything into the command line. I can see the PWD of the step is the root of the project, so I can work everything off that for reading. Thankyou for your help!

markjfisher commented 1 year ago

I've got my first working 6502 code working in a test!

image

This is great. I've written a few helper steps as you can see to convert from MADS assembler labels to ACME, so that the other steps can still function correctly, and I was able to easily load the exe into correct memory location by reading data from the file itself, as it defines the load address in the first 6 bytes, so overall I've got a great place to start.

martinpiper commented 1 year ago

That's great. :)

markjfisher commented 1 year ago

My first real feature, using macros to help with boiler plate that tests a decompress library. https://github.com/markjfisher/fujinet-config-ng/blob/master/testing/bdd-testing/features/decompress/decompress.feature

I think I'll close this issue as complete for now. I have enough to get on with. It's a little slow at times because I'd really like to have:

also I don't have too much familiarity with the steps you've already created, and find myself hunting through features etc for ideas, but that'll come with more tests.

martinpiper commented 1 year ago

IntelliJ main menu->Run-Attach To Process

markjfisher commented 1 year ago

In all the years I've been using IntelliJ, never seen that, always invoked it from debug. Thanks