Failed to load `experiment.xml` with Ubuntu and `v3xt.jar`

[kinow]

From https://github.com/openjournals/joss-reviews/issues/2791

I'm getting the following when trying to load the sample experiment.

$ ~/Development/java/jdk-11.0.2/bin/java -jar v3xt.jar 
Exception when attempting to load/save:/home/kinow/Downloads/1.1/experiment.xml
java.io.IOException: Could not read System File: data\systems\JabRef\3.7\jabref-3_7.sysmdl
java.io.IOException: Could not read System File: data\systems\JabRef\3.7\jabref-3_7.sysmdl
    at se.lnu.siq.s4rdm3x.experiments.system.FileBased.<init>(FileBased.java:32)
    at se.lnu.siq.s4rdm3x.experiments.ExperimentXMLPersistence.elementToRunner(ExperimentXMLPersistence.java:149)
    at se.lnu.siq.s4rdm3x.experiments.ExperimentXMLPersistence.loadExperimentRunners(ExperimentXMLPersistence.java:74)
    at experimenting.ExperimentsView.doSaveButtons(ExperimentsView.java:225)
    at experimenting.ExperimentsView.doView(ExperimentsView.java:116)
    at Main.mainLoop(Main.java:181)
    at Main.lambda$run$0(Main.java:144)
    at uno.glfw.GlfwWindow.loop(GlfwWindow.kt:654)
    at Main.run(Main.java:135)
    at Main.main(Main.java:54)

Any idea what I got wrong?

[kinow]

I am using the release 1.2 (even though the directory says 1.1). And the file that it's failing to load appears to exist in the expected location. Might be some issue with how the file is loaded, or maybe it works on Mac/Win but not Linux, or maybe it's something else with my environment?

[h0bb3]

A likely error is that the example experiment file uses windows type file delimiters (as it was saved on my windows dev machine).

The line "" should be changed to:

If you can confirm that this is the problem, I can make some code changes to save the file path in a system-neutral way when the experiments are saved.

[kinow]

I still have the project open here in Eclipse. Let me just finish working on another project, and I will try to locate the code, change, test, and if that works will send a PR :+1:

[h0bb3]

cool!

[kinow]

@tobias-dv-lnu I think the code on master might be working. When I loaded the ExperimentXMLPersistence.java class in Eclipse, I realized it was taking care of replacing some characters by the File.fileSeparator as in here.

I debugged the jar from Eclipse (-Xdebug... specifying the remote port) after checking from the tag, and confirmed the path is indeed using the wrong separator.

I don't know how to generate the two jars with Gradle @tobias-dv-lnu , so I'm unable to modify the source code and confirm it's really working from master. Maybe with a new release we can assess it again?

Thanks B

[h0bb3]

Yes, you are correct.

It seems there can be an attribute "file_separator" in the XML that should specify the separator used when the file was saved, it should then be replaced with the platform-specific separator File.separator

The experiment XML in the release seems to be missing this attribute, however, as per the code it should default to the more odd windows specific separator . I.e. \ should be replaced with / on Linux.

I will investigate and create a new release asap.

[h0bb3]

I have confirmed the code working in cmdexrunner (it uses the same code but is command line based) on an ubuntu-machine. I will create a new release.

Edit: new release is available https://github.com/tobias-dv-lnu/s4rdm3x/releases

tested cmdexrunner with the experiments.xml on Ubuntu and it worked. Hopefully, v3xt will also work.

[kinow]

Thanks for preparing this new release so quickly @tobias-dv-lnu ! I will tryt o restart the review tonight (NZ time) and confirm the error is gone / close this issue.

[kinow]

I started v3xt.jar, then clicked on the "Load" button.

$ ~/Development/java/jdk-15/bin/java -jar v3xt.jar 
Could not add system model: data/systems/argouml/argouml.sysmdl - referenced jars do not exist
Could not add system model: data/systems/ant/ant.sysmdl - referenced jars do not exist
Could not add system model: data/systems/JabRef/3.7/jabref-3_7.sysmdl - referenced jars do not exist
Could not add system model: data/systems/sweethome3d/sweethome3d.sysmdl - referenced jars do not exist
Could not add system model: data/systems/ProM6.9/ProM_6_9.sysmdl - referenced jars do not exist
Could not add system model: data/systems/teammates/teammates.sysmdl - referenced jars do not exist
Could not add system model: data/systems/lucene/lucene.sysmdl - referenced jars do not exist

This is a different box, but also with Ubuntu latest LTS (20.04).

[h0bb3]

Yes, that would be normal as you also need to download the systems to analyze (I cannot include them in the distribution due to licensing).

I suggest you download JabRef 3.7 jar and place it in the same directory as the jabref-3_7.sysymdl JabRef is what is used in the included experiment.

You can find it here: https://github.com/JabRef/jabref/releases/tag/v3.7

This is mentioned in the documentation under Running/Data Systems. A better error message would also be good.

[kinow]

Hi @tobias-dv-lnu

I think I'm understanding more how s4rdm3x works. From what I understood, there are several projects in the systems folder. I thought they were dependencies for the experiments, but looks like they are projects which you can run the experiments against?

Which means if I wanted to test Apache Commons Lang, I would need to create a folder data/systems/commons-lang, and create a commons-lang.sysmdl file with the details of the project architecture.

Is that correct?

This is mentioned in the documentation under Running/Data Systems. A better error message would also be good.

I think we could just move or link the sections. I am following the docs from the top to the bottom. Looking one section at a time. I saw in one section that I could run the v3xt.jar, and immediately started playing with it.

Maybe in the end of that section, we could mention that users are expected to read the next section before running/loading the existing experiments. Something like "In order to run the experiments you will have to first download at least one of the projects used. We will see how this can be done in the next section etc etc", or something along these lines?

WDYT?

[h0bb3]

Hey!

I think I'm understanding more how s4rdm3x works. From what I understood, there are several projects in the systems folder. I thought they were dependencies for the experiments, but looks like they are projects which you can run the experiments against?

Exactly!

_Which means if I wanted to test Apache Commons Lang, I would need to create a folder data/systems/commons-lang, and create a commons-lang.sysmdl file with the details of the project architecture.

Is that correct?_

Correct! The .sysmdl are quite simple text files, so you can create one in a text editor. The problem is, of course, that you need a mapping for the source code to the architectural modules these are often created by experts or at least need a large effort and deep knowledge of the system. From what I know I have the largest number of such models included in this repo.

You are correct about the documentation, I guess most would read from top to bottom and then start to play with things. I will make some changes.

Thanks!

[kinow]

I loaded the experiment.xml again, after putting the jar in the folder of JabRef as you said @tobias-dv-lnu . It printed the errors for the other applications, but I am ignoring them for now.

I also pressed "Run Experiment". There was no clear indication that the experiment was running successfully for JabRef (or not one that was clear to me?). Is there somewhere I could accompany via logs or GUI what's happening?

After a while I stopped with the button Stop Experiment.

And finally launched the cmdexrunner.jar:

$ ~/Development/java/jdk-11.0.2/bin/java -jar cmdexrunner.jar -experiment experiment.xml -outDir bruno
Running 1 threads. Saving data in: bruno/JabRef_37_1_0
Running experiments: 
All experiment threads Started!
0, WARNING: core mtj jar files are not available as resources to this classloader (jdk.internal.loader.ClassLoaders$AppClassLoader@799f7e29)

Are these the correct steps @tobias-dv-lnu ? I am trying to get a quick overview of s4rdm3x, and then read the code and paper after I understood the basic operation of the tool.

The cmdexrunner.jar produced an CSV with some metrics that I plan to take a closer look later. Screenshot of the CSV below.

My intention is to run s4rdm3x against Apache Commons Lang. I remember using a tool that appears to be similar (JMack? MackJ?). I remember I could define the packages/modules of my code, and how they should relate to each other. That tool would then run a static analysis and give me the report saying whether there were architectural issues or not. Though I still need to read more about the theory behind s4rdm3x before I can compare both…

[h0bb3]

You should get some results from the experiments in the plots windows (however, as default they may be hidden or very small). Make sure JabRef is checked before you start the experiment (it should be but you never know). And then check/enlarge the plot windows. You should see some results after a few seconds.

[h0bb3]

Interesting case with Apache, let me know if/how I can help.

In the research community, the main tools used for static architectural conformance checking are JabRef (the sysmdl format is based on an older version of the JabRef format) and Hussact, but there are also tools/experiments that analyze source code and try to find problems or abstractions. For some reason, JMack/MackJ sounds familiar but I cannot recall exactly where I heard it.

I'm mainly working on trying to automate the mapping of source code to the architectural modules as this is a prerequisite to doing static architectural conformance checking. s4rdm3x is about testing approaches to automating such mappings and comparing with a human made mapping (that is why a ground truth mapping is needed).

[h0bb3]

FYI: I moved the Data systems part of the documentation to be a sub-section to Installation. I.e. first you download the program, then you get instructions on the data systems needed, and finally instructions on how to run.

Much better imho :)

[kinow]

Installation instructions working @tobias-dv-lnu . Successfully ran my first experiment with s4rdm3x (-:

Thanks for the help!

[h0bb3]

Yay, so glad you got it working. :)