pderakhshanfar
commented
5 years ago @vmassol,
Currently, EvoSuite and Botsing, on top of that, do not support JUnit 5.
Open vmassol opened 5 years ago
pderakhshanfar
commented
5 years ago @vmassol,
Currently, EvoSuite and Botsing, on top of that, do not support JUnit 5.
vmassol
commented
5 years ago Currently, EvoSuite and Botsing, on top of that, do not support JUnit 5.
Ouch so it means we cannot run it on XWiki, at least not on the majority of modules which now have JUnit 5 tests (and more converted every day) :(
pderakhshanfar
commented
5 years ago We will see how we can add JUnit 5 support. However, for now, you can use only the static analysis of the model generator.
vmassol
commented
5 years ago However, for now, you can use only the static analysis of the model generator.
Could you tell me what this means and what I need to change from my list of steps above? Thanks!
pderakhshanfar
commented
5 years ago The model generator has two phases: 1- static analysis for finding the existing call sequences in the source code; 2- dynamic analysis for executing the existing test cases and carving the call sequences from them. Both of these phases are enabled by default. So, with the model seeding command line that you mentioned in this issue, you should already have models from the call sequences which are produced by the static analyzer.
vmassol
commented
5 years ago ok, so I'll carry on (i.e. without the dynamic analysis) and see what I get, thanks.
vmassol
commented
5 years ago @pderakhshanfar is it normal that my target/botsing/models directory is empty? (i.e. no models)
vmassol
commented
5 years ago I've tried executing the following:
java -d64 -Xmx4000m -jar ~/.m2/repository/org/evosuite/evosuite-master/1.0.7-SNAPSHOT/evosuite-master-1.0.7-SNAPSHOT.jar \
-prefix "org.xwiki.xml" \
-projectCP "$xwikicp" \
-generateMOSuite \
-Dalgorithm=DynaMOSA \
-Dsearch_budget=60 \
-Dseed_clone="0.5" \
-Donline_model_seeding=TRUE \
-Dmodel_path="target/botsing/models" \
-Dtest_dir="target/botsing/evosuite-tests" \
-Dreport_dir="target/botsing/evosuite-report" \
-Dno_runtime_dependency=trueAnd got:
vmassol ~/dev/xwiki/xwiki-commons/xwiki-commons-core/xwiki-commons-xml (master *+) $ java -d64 -Xmx4000m -jar ~/.m2/repository/org/evosuite/evosuite-master/1.0.7-SNAPSHOT/evosuite-master-1.0.7-SNAPSHOT.jar \
> -prefix "org.xwiki.xml" \
> -projectCP "$xwikicp" \
> -generateMOSuite \
> -Dalgorithm=DynaMOSA \
> -Dsearch_budget=60 \
> -Dseed_clone="0.5" \
> -Donline_model_seeding=TRUE \
> -Dmodel_path="target/botsing/models" \
> -Dtest_dir="target/botsing/evosuite-tests" \
> -Dreport_dir="target/botsing/evosuite-report" \
> -Dno_runtime_dependency=true
entry: /Users/vmassol/.m2/repository/org/jdom/jdom/1.1.3/jdom-1.1.3.jar
...
** CP: /Users/vmassol/.m2/repository/org/jdom/jdom/1.1.3/jdom-1.1.3.jar:...
* EvoSuite 1.0.7-SNAPSHOT
* Analyzing classpath (generating inheritance tree)
- /Users/vmassol/.m2/repository/org/jdom/jdom/1.1.3/jdom-1.1.3.jar
...
* Found 0 matching classes for prefix org.xwiki.xmlAny idea? Thx
pderakhshanfar
commented
5 years ago @pderakhshanfar is it normal that my
target/botsing/modelsdirectory is empty? (i.e. no models)
No the model should not be empty. Are you sure that the passed bytecodes contain the classes with the given prefix?
vmassol ~/dev/xwiki/xwiki-commons/xwiki-commons-core/xwiki-commons-xml (master *+) $ java -d64 -Xmx4000m -jar ~/.m2/repository/org/evosuite/evosuite-master/1.0.7-SNAPSHOT/evosuite-master-1.0.7-SNAPSHOT.jar \
-prefix "org.xwiki.xml" \ -projectCP "$xwikicp" \ -generateMOSuite \ -Dalgorithm=DynaMOSA \ -Dsearch_budget=60 \ -Dseed_clone="0.5" \ -Donline_model_seeding=TRUE \ -Dmodel_path="target/botsing/models" \ -Dtest_dir="target/botsing/evosuite-tests" \ -Dreport_dir="target/botsing/evosuite-report" \ -Dno_runtime_dependency=true
evosuite-master-1.0.7-SNAPSHOT.jar should not have -prefix input parameter right? It should be -class <class-name>
vmassol
commented
5 years ago No the model should not be empty. Are you sure that the passed bytecodes contain the classes with the given prefix?
Yes I'm sure, see: https://github.com/xwiki/xwiki-commons/tree/master/xwiki-commons-core/xwiki-commons-xml/src/main/java/org/xwiki/xml
evosuite-master-1.0.7-SNAPSHOT.jar should not have -prefix input parameter right? It should be -class
If you're asking me, I don't know :) I was just following the tutorial at https://github.com/STAMP-project/evosuite-model-seeding-tutorial which says:
-class "org.ow2.authzforce.core.pdp.impl.PdpBean" define the class under test for which test cases will be generated. It is also possible to generate a test suite for a whole package using the -prefix "org.ow2.authzforce.core.pdp.impl" option (see EvoSuite command line documentation for more details);
Since I didn't want to generate tests just for a single class but for all classes under the org.xwiki.xml.* packages, I selected to use the -prefix flag. Is that wrong?
vmassol
commented
5 years ago I've now tried with:
java -d64 -Xmx4000m -jar ~/.m2/repository/org/evosuite/evosuite-master/1.0.7-SNAPSHOT/evosuite-master-1.0.7-SNAPSHOT.jar \
-class "org.xwiki.xml.XMLUtilsTest" \
-projectCP "$xwikicp" \
-generateMOSuite \
-Dalgorithm=DynaMOSA \
-Dsearch_budget=60 \
-Dseed_clone="0.5" \
-Donline_model_seeding=TRUE \
-Dmodel_path="target/botsing/models" \
-Dtest_dir="target/botsing/evosuite-tests" \
-Dreport_dir="target/botsing/evosuite-report" \
-Dno_runtime_dependency=trueAnd I get:
...
* EvoSuite 1.0.7-SNAPSHOT
* Going to generate test cases for class: org.xwiki.xml.XMLUtilsTest
* Unknown class: org.xwiki.xml.XMLUtilsTest. Be sure its full qualifying name is correct and the classpath is properly set with '-projectCP'The CP is correct IMO:
vmassol ~/dev/xwiki/xwiki-commons/xwiki-commons-core/xwiki-commons-xml (master *+) $ echo $xwikicp
/Users/vmassol/.m2/repository/org/jdom/jdom/1.1.3/jdom-1.1.3.jar:/Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-component-api/11.9-SNAPSHOT/xwiki-commons-component-api-11.9-SNAPSHOT.jar:/Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-stability/11.9-SNAPSHOT/xwiki-commons-stability-11.9-SNAPSHOT.jar:/Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-text/11.9-SNAPSHOT/xwiki-commons-text-11.9-SNAPSHOT.jar:/Users/vmassol/.m2/repository/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar:/Users/vmassol/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:/Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-context/11.9-SNAPSHOT/xwiki-commons-context-11.9-SNAPSHOT.jar:/Users/vmassol/.m2/repository/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar:/Users/vmassol/.m2/repository/org/apache/commons/commons-text/1.8/commons-text-1.8.jar:/Users/vmassol/.m2/repository/net/sourceforge/htmlcleaner/htmlcleaner/2.22/htmlcleaner-2.22.jar:/Users/vmassol/.m2/repository/org/jdom/jdom2/2.0.6/jdom2-2.0.6.jar:/Users/vmassol/.m2/repository/xerces/xercesImpl/2.12.0/xercesImpl-2.12.0.jar:/Users/vmassol/.m2/repository/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.jar:/Users/vmassol/.m2/repository/net/java/dev/stax-utils/stax-utils/20070216/stax-utils-20070216.jar:/Users/vmassol/.m2/repository/com/fasterxml/woodstox/woodstox-core/6.0.1/woodstox-core-6.0.1.jar:/Users/vmassol/.m2/repository/org/codehaus/woodstox/stax2-api/4.2/stax2-api-4.2.jar:/Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-tool-test-component/11.9-SNAPSHOT/xwiki-commons-tool-test-component-11.9-SNAPSHOT.jar:/Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-tool-test-simple/11.9-SNAPSHOT/xwiki-commons-tool-test-simple-11.9-SNAPSHOT.jar:/Users/vmassol/.m2/repository/junit/junit/4.12/junit-4.12.jar:/Users/vmassol/.m2/repository/org/hamcrest/hamcrest-core/2.1/hamcrest-core-2.1.jar:/Users/vmassol/.m2/repository/org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar:/Users/vmassol/.m2/repository/org/junit/jupiter/junit-jupiter/5.5.2/junit-jupiter-5.5.2.jar:/Users/vmassol/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.5.2/junit-jupiter-api-5.5.2.jar:/Users/vmassol/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar:/Users/vmassol/.m2/repository/org/junit/platform/junit-platform-commons/1.5.2/junit-platform-commons-1.5.2.jar:/Users/vmassol/.m2/repository/org/junit/jupiter/junit-jupiter-params/5.5.2/junit-jupiter-params-5.5.2.jar:/Users/vmassol/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.5.2/junit-jupiter-engine-5.5.2.jar:/Users/vmassol/.m2/repository/org/junit/platform/junit-platform-launcher/1.5.2/junit-platform-launcher-1.5.2.jar:/Users/vmassol/.m2/repository/org/apiguardian/apiguardian-api/1.1.0/apiguardian-api-1.1.0.jar:/Users/vmassol/.m2/repository/org/junit/platform/junit-platform-engine/1.5.2/junit-platform-engine-1.5.2.jar:/Users/vmassol/.m2/repository/org/junit/vintage/junit-vintage-engine/5.5.2/junit-vintage-engine-5.5.2.jar:/Users/vmassol/.m2/repository/org/hamcrest/hamcrest-library/2.1/hamcrest-library-2.1.jar:/Users/vmassol/.m2/repository/org/jmock/jmock/2.6.0/jmock-2.6.0.jar:/Users/vmassol/.m2/repository/org/jmock/jmock-junit4/2.6.0/jmock-junit4-2.6.0.jar:/Users/vmassol/.m2/repository/commons-io/commons-io/2.6/commons-io-2.6.jar:/Users/vmassol/.m2/repository/org/mockito/mockito-core/3.1.0/mockito-core-3.1.0.jar:/Users/vmassol/.m2/repository/net/bytebuddy/byte-buddy/1.10.1/byte-buddy-1.10.1.jar:/Users/vmassol/.m2/repository/net/bytebuddy/byte-buddy-agent/1.9.10/byte-buddy-agent-1.9.10.jar:/Users/vmassol/.m2/repository/org/objenesis/objenesis/2.6/objenesis-2.6.jar:/Users/vmassol/.m2/repository/org/mockito/mockito-junit-jupiter/3.1.0/mockito-junit-jupiter-3.1.0.jar:/Users/vmassol/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar:/Users/vmassol/.m2/repository/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar:/Users/vmassol/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar:/Users/vmassol/.m2/repository/org/slf4j/log4j-over-slf4j/1.7.28/log4j-over-slf4j-1.7.28.jar:/Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-observation-api/11.9-SNAPSHOT/xwiki-commons-observation-api-11.9-SNAPSHOT.jar:/Users/vmassol/.m2/repository/jmock/jmock/1.2.0/jmock-1.2.0.jar:/Users/vmassol/.m2/repository/jmock/jmock-cglib/1.2.0/jmock-cglib-1.2.0.jar:/Users/vmassol/.m2/repository/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.jar:/Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-component-default/11.9-SNAPSHOT/xwiki-commons-component-default-11.9-SNAPSHOT.jar:/Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-component-observation/11.9-SNAPSHOT/xwiki-commons-component-observation-11.9-SNAPSHOT.jar:/Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-configuration-api/11.9-SNAPSHOT/xwiki-commons-configuration-api-11.9-SNAPSHOT.jar:/Users/vmassol/.m2/repository/commons-beanutils/commons-beanutils/1.9.4/commons-beanutils-1.9.4.jar:/Users/vmassol/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jarThe CP is correct IMO
Ah no.... it's the CP for dependencies. It's missing target/classes and target/test-classes. Retrying
vmassol
commented
5 years ago Cool so that was the problem. I now get models and evosuite is doing something :)
FTR the right process is:
target/classes and target/test-classestoo:export xwikicp="target/classes:target/test-classses:/Users/vmassol/.m2/repository/org/jdom/jdom/1.1.3/jdom-1.1.3.jar:..."ok I got some test generated for -class "org.xwiki.xml.XMLUtilsTest", so that's a first step :)
I've checked the test though and it doesn't seem interesting at all and it's actually missing assertions so my guess (didn't verify it) is that it's going to reduce the mutation score we're trying to increase ;)
/*
* This file was automatically generated by EvoSuite
* Thu Oct 03 15:22:48 GMT 2019
*/
package org.xwiki.xml;
import org.junit.Test;
import static org.junit.Assert.*;
import org.opentest4j.AssertionFailedError;
import org.xwiki.xml.XMLUtilsTest;
public class XMLUtilsTest_ESTest {
@Test(timeout = 4000)
public void test00() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.unescapeXMLComment();
}
@Test(timeout = 4000)
public void test01() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
// Undeclared exception!
try {
xMLUtilsTest0.escapeApos();
fail("Expecting exception: AssertionFailedError");
} catch(AssertionFailedError e) {
//
// ' wrongly escaped to non-HTML ' ==> expected: <false> but was: <true>
//
}
}
@Test(timeout = 4000)
public void test02() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.escapeFAttributeValueWithNull();
}
@Test(timeout = 4000)
public void test03() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.unescapeWithNull();
}
@Test(timeout = 4000)
public void test04() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.escapeElementContentNonAscii();
}
@Test(timeout = 4000)
public void test05() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.escapeElementContentEmptyString();
}
@Test(timeout = 4000)
public void test06() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.escape();
}
@Test(timeout = 4000)
public void test07() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
// Undeclared exception!
try {
xMLUtilsTest0.escapeAttributeValueApos();
fail("Expecting exception: AssertionFailedError");
} catch(AssertionFailedError e) {
//
// ' wrongly escaped to non-HTML ' ==> expected: <false> but was: <true>
//
}
}
@Test(timeout = 4000)
public void test08() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.escapeElementContent();
}
@Test(timeout = 4000)
public void test09() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.unescapeEmptyString();
}
@Test(timeout = 4000)
public void test10() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.escapeAttributeValue();
}
@Test(timeout = 4000)
public void test11() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.escapeWithNull();
}
@Test(timeout = 4000)
public void test12() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.escapeXMLComment();
}
@Test(timeout = 4000)
public void test13() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.createDomDocument();
}
@Test(timeout = 4000)
public void test14() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.escapeFAttributeValueEmptyString();
}
@Test(timeout = 4000)
public void test15() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.escapeAttributeValueNonAscii();
}
@Test(timeout = 4000)
public void test16() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.escapeElementContentWithNull();
}
@Test(timeout = 4000)
public void test17() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.unescape();
}
@Test(timeout = 4000)
public void test18() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.escapeEmptyString();
}
@Test(timeout = 4000)
public void test19() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.escapeNonAscii();
}
@Test(timeout = 4000)
public void test20() throws Throwable {
XMLUtilsTest xMLUtilsTest0 = new XMLUtilsTest();
xMLUtilsTest0.unescapeOtherEscapes();
}
}FTR original test: https://github.com/xwiki/xwiki-commons/blob/af2a7d6cad13a5fc7caece99f9c3f4fbcce29743/xwiki-commons-core/xwiki-commons-xml/src/test/java/org/xwiki/xml/XMLUtilsTest.java
vmassol
commented
5 years ago Erratum: I've wrongly used -class "org.xwiki.xml.XMLUtilsTest". I've now retested with -class "org.xwiki.xml.XMLUtils" and it seems better:
/*
* This file was automatically generated by EvoSuite
* Thu Oct 03 16:13:50 GMT 2019
*/
package org.xwiki.xml;
import org.junit.Test;
import static org.junit.Assert.*;
import static org.evosuite.shaded.org.mockito.Mockito.*;
import java.io.InputStream;
import java.io.SequenceInputStream;
import java.util.Enumeration;
import javax.xml.transform.Source;
import javax.xml.transform.TransformerException;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.sax.SAXSource;
import javax.xml.transform.stream.StreamSource;
import org.apache.html.dom.HTMLDocumentImpl;
import org.apache.wml.dom.WMLDocumentImpl;
import org.apache.wml.dom.WMLPElementImpl;
import org.apache.xerces.dom.DOMInputImpl;
import org.apache.xerces.dom.DeferredDocumentImpl;
import org.apache.xerces.dom.ElementImpl;
import org.apache.xerces.dom.TextImpl;
import org.evosuite.runtime.ViolatedAssumptionAnswer;
import org.evosuite.runtime.testdata.EvoSuiteFile;
import org.w3c.dom.Document;
import org.w3c.dom.DocumentType;
import org.w3c.dom.Node;
import org.w3c.dom.Text;
import org.xwiki.xml.XMLUtils;
public class XMLUtils_ESTest {
@Test(timeout = 4000)
public void test00() throws Throwable {
Document document0 = XMLUtils.createDOMDocument();
// Undeclared exception!
try {
XMLUtils.extractXML(document0, (-2295), (-2295));
fail("Expecting exception: RuntimeException");
} catch(RuntimeException e) {
//
// Failed to extract XML
//
}
}
@Test(timeout = 4000)
public void test01() throws Throwable {
try {
XMLUtils.formatXMLContent("K$#OB[dvAT=+4,fD");
fail("Expecting exception: TransformerException");
} catch(TransformerException e) {
}
}
@Test(timeout = 4000)
public void test02() throws Throwable {
Object object0 = new Object();
String string0 = XMLUtils.escape(object0);
assertNotNull(string0);
}
@Test(timeout = 4000)
public void test03() throws Throwable {
DeferredDocumentImpl deferredDocumentImpl0 = new DeferredDocumentImpl(false, false);
// Undeclared exception!
try {
XMLUtils.extractXML(deferredDocumentImpl0, 64, 64);
fail("Expecting exception: RuntimeException");
} catch(RuntimeException e) {
//
// Failed to extract XML
//
}
}
@Test(timeout = 4000)
public void test04() throws Throwable {
String string0 = XMLUtils.escapeAttributeValue((Object) null);
assertNull(string0);
}
@Test(timeout = 4000)
public void test05() throws Throwable {
String string0 = XMLUtils.escapeAttributeValue("J\"pb_6n&@aF-d`V");
assertEquals("J"pb_6n&@aF-d`V", string0);
assertNotNull(string0);
}
@Test(timeout = 4000)
public void test06() throws Throwable {
String string0 = XMLUtils.escapeAttributeValue("^)'Ct< TT9S<z?@U");
assertNotNull(string0);
assertEquals("^)'Ct< TT9S<z?@U", string0);
}
@Test(timeout = 4000)
public void test07() throws Throwable {
String string0 = XMLUtils.escapeAttributeValue("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<K$#OB[dvAT=+4,fD/>");
assertNotNull(string0);
assertEquals("<?xml version="1.0" encoding="UTF-8"?>\n<K$#OB[dvAT=+4,fD/>", string0);
}
@Test(timeout = 4000)
public void test08() throws Throwable {
String string0 = XMLUtils.escapeAttributeValue("{http://xml.apache.org/xslt}indent-amount");
assertNotNull(string0);
assertEquals("{http://xml.apache.org/xslt}indent-amount", string0);
}
@Test(timeout = 4000)
public void test09() throws Throwable {
String string0 = XMLUtils.escapeElementContent("J\"pb_6n&@aF-d`V");
assertNotNull(string0);
assertEquals("J\"pb_6n&@aF-d`V", string0);
}
@Test(timeout = 4000)
public void test10() throws Throwable {
String string0 = XMLUtils.escapeElementContent((Object) null);
assertNull(string0);
}
@Test(timeout = 4000)
public void test11() throws Throwable {
String string0 = XMLUtils.escapeElementContent("a<vRY:gk6Dw>$86f/");
assertEquals("a<vRY:gk6Dw>$86f/", string0);
assertNotNull(string0);
}
@Test(timeout = 4000)
public void test12() throws Throwable {
Object object0 = new Object();
String string0 = XMLUtils.unescape(object0);
assertNotNull(string0);
}
@Test(timeout = 4000)
public void test13() throws Throwable {
String string0 = XMLUtils.unescape((Object) null);
assertNull(string0);
}
@Test(timeout = 4000)
public void test14() throws Throwable {
DOMInputImpl dOMInputImpl0 = new DOMInputImpl();
Document document0 = XMLUtils.parse(dOMInputImpl0);
assertNull(document0);
}
@Test(timeout = 4000)
public void test15() throws Throwable {
TextImpl textImpl0 = new TextImpl();
String string0 = XMLUtils.serialize((Node) textImpl0, false);
assertEquals("", string0);
}
@Test(timeout = 4000)
public void test16() throws Throwable {
String string0 = XMLUtils.transform((Source) null, (Source) null);
assertNull(string0);
}
@Test(timeout = 4000)
public void test17() throws Throwable {
SAXSource sAXSource0 = new SAXSource();
String string0 = XMLUtils.transform(sAXSource0, (Source) null);
assertNull(string0);
}
@Test(timeout = 4000)
public void test18() throws Throwable {
String string0 = XMLUtils.escapeXMLComment(")VsE\"K <dTErL{N#c-");
assertEquals(")VsE\"K <dTErL{N#c-\\", string0);
}
@Test(timeout = 4000)
public void test19() throws Throwable {
String string0 = XMLUtils.unescapeXMLComment(")VsE\"K <\\dTErL{N#c-");
assertEquals(")VsE\"K <dTErL{N#c-", string0);
}
@Test(timeout = 4000)
public void test20() throws Throwable {
String string0 = XMLUtils.serialize((Node) null, false);
assertEquals("", string0);
}
@Test(timeout = 4000)
public void test21() throws Throwable {
DeferredDocumentImpl deferredDocumentImpl0 = new DeferredDocumentImpl(false, false);
String string0 = XMLUtils.serialize((Node) deferredDocumentImpl0, false);
assertEquals("", string0);
}
@Test(timeout = 4000)
public void test22() throws Throwable {
WMLDocumentImpl wMLDocumentImpl0 = mock(WMLDocumentImpl.class, new ViolatedAssumptionAnswer());
doReturn((DocumentType) null).when(wMLDocumentImpl0).getDoctype();
doReturn((String) null).when(wMLDocumentImpl0).getInputEncoding();
doReturn((String) null, (String) null).when(wMLDocumentImpl0).getXmlEncoding();
doReturn((String) null).when(wMLDocumentImpl0).getXmlVersion();
WMLPElementImpl wMLPElementImpl0 = new WMLPElementImpl(wMLDocumentImpl0, "");
String string0 = XMLUtils.serialize((Node) wMLPElementImpl0, true);
assertEquals("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n</>", string0);
}
@Test(timeout = 4000)
public void test23() throws Throwable {
// Undeclared exception!
try {
XMLUtils.escapeXMLComment((String) null);
fail("Expecting exception: NullPointerException");
} catch(NullPointerException e) {
//
// no message in exception (getMessage() returned null)
//
}
}
@Test(timeout = 4000)
public void test24() throws Throwable {
// Undeclared exception!
try {
XMLUtils.formatXMLContent((String) null);
fail("Expecting exception: NullPointerException");
} catch(NullPointerException e) {
//
// no message in exception (getMessage() returned null)
//
}
}
@Test(timeout = 4000)
public void test25() throws Throwable {
// Undeclared exception!
try {
XMLUtils.unescapeXMLComment((String) null);
fail("Expecting exception: NullPointerException");
} catch(NullPointerException e) {
//
// no message in exception (getMessage() returned null)
//
}
}
@Test(timeout = 4000)
public void test26() throws Throwable {
String string0 = XMLUtils.escape("");
assertEquals("", string0);
}
@Test(timeout = 4000)
public void test27() throws Throwable {
String string0 = XMLUtils.escape((Object) null);
assertNull(string0);
}
@Test(timeout = 4000)
public void test28() throws Throwable {
String string0 = XMLUtils.escapeAttributeValue("");
assertEquals("", string0);
}
@Test(timeout = 4000)
public void test29() throws Throwable {
String string0 = XMLUtils.escapeElementContent("");
assertEquals("", string0);
}
@Test(timeout = 4000)
public void test30() throws Throwable {
String string0 = XMLUtils.escapeXMLComment("");
assertEquals("", string0);
}
@Test(timeout = 4000)
public void test31() throws Throwable {
HTMLDocumentImpl hTMLDocumentImpl0 = new HTMLDocumentImpl();
Text text0 = hTMLDocumentImpl0.createTextNode("");
String string0 = XMLUtils.extractXML(text0, 9, 2116);
assertEquals("", string0);
}
@Test(timeout = 4000)
public void test32() throws Throwable {
DeferredDocumentImpl deferredDocumentImpl0 = new DeferredDocumentImpl(true, true);
ElementImpl elementImpl0 = new ElementImpl(deferredDocumentImpl0, "K$#OB[dvAT=+4,fD");
XMLUtils.serialize((Node) elementImpl0);
String string0 = XMLUtils.serialize((Node) elementImpl0);
assertEquals("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<K$#OB[dvAT=+4,fD/>", string0);
}
@Test(timeout = 4000)
public void test33() throws Throwable {
String string0 = XMLUtils.unescape("");
assertEquals("", string0);
}
@Test(timeout = 4000)
public void test34() throws Throwable {
String string0 = XMLUtils.unescapeXMLComment("");
assertEquals("", string0);
}
@Test(timeout = 4000)
public void test35() throws Throwable {
String string0 = XMLUtils.escapeXMLComment("-5z)oJ_iNE");
assertEquals("-5z)oJ_iNE", string0);
}
@Test(timeout = 4000)
public void test36() throws Throwable {
Enumeration<InputStream> enumeration0 = (Enumeration<InputStream>) mock(Enumeration.class, new ViolatedAssumptionAnswer());
doReturn(false).when(enumeration0).hasMoreElements();
SequenceInputStream sequenceInputStream0 = new SequenceInputStream(enumeration0);
DOMSource dOMSource0 = new DOMSource();
StreamSource streamSource0 = new StreamSource(sequenceInputStream0);
String string0 = XMLUtils.transform(streamSource0, dOMSource0);
assertNull(string0);
}
}I now need to analyze it to see how useful it is.
vmassol
commented
5 years ago Question: how do I integrate the generated test class in my test sources? I'm asking because it's not compiling.
See the red parts in the following screenshot:
I guess there's some evosuite runtime library to depend on. Could you give me the URL of where it can be found in a maven repo? Thx!
pderakhshanfar
commented
5 years ago Question: how do I integrate the generated test class in my test sources? I'm asking because it's not compiling.
See the red parts in the following screenshot:
I guess there's some evosuite runtime library to depend on. Could you give me the URL of where it can be found in a maven repo? Thx!
To generate tests without evosuite dependencies, add the following command parameter:
-Dno_runtime_dependency=true
Also, the evosuite dependency can be found here.
vmassol
commented
5 years ago Hi @pderakhshanfar
Thanks. However, as you can see on https://github.com/STAMP-project/botsing/issues/89#issuecomment-537991272 I've already used -Dno_runtime_dependency=true to generate the tests above they still require some dependency. Is that a bug?
Thanks
vmassol
commented
5 years ago @pderakhshanfar I've regenerated the model today (since I had cleaned my module target's directory) and then executed evosuite again but I now get the following error that I don't understand:
vmassol ~/dev/xwiki/xwiki-commons/xwiki-commons-core/xwiki-commons-xml (master *+) $ java -d64 -Xmx4000m -jar ~/.m2/repository/org/evosuite/evosuite-master/1.0.7-SNAPSHOT/evosuite-master-1.0.7-SNAPSHOT.jar \
> -class "org.xwiki.xml.XMLUtils" \
> -projectCP "$xwikicp" \
> -generateMOSuite \
> -Dalgorithm=DynaMOSA \
> -Dsearch_budget=60 \
> -Dseed_clone="0.5" \
> -Donline_model_seeding=TRUE \
> -Dmodel_path="target/botsing/models" \
> -Dtest_dir="target/botsing/evosuite-tests" \
> -Dreport_dir="target/botsing/evosuite-report" \
> -Dno_runtime_dependency=true
entry: target/classes
entry: target/test-classses
entry: /Users/vmassol/.m2/repository/org/jdom/jdom/1.1.3/jdom-1.1.3.jar
entry: /Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-component-api/11.9-SNAPSHOT/xwiki-commons-component-api-11.9-SNAPSHOT.jar
entry: /Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-stability/11.9-SNAPSHOT/xwiki-commons-stability-11.9-SNAPSHOT.jar
entry: /Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-text/11.9-SNAPSHOT/xwiki-commons-text-11.9-SNAPSHOT.jar
entry: /Users/vmassol/.m2/repository/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar
entry: /Users/vmassol/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar
entry: /Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-context/11.9-SNAPSHOT/xwiki-commons-context-11.9-SNAPSHOT.jar
entry: /Users/vmassol/.m2/repository/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar
entry: /Users/vmassol/.m2/repository/org/apache/commons/commons-text/1.8/commons-text-1.8.jar
entry: /Users/vmassol/.m2/repository/net/sourceforge/htmlcleaner/htmlcleaner/2.22/htmlcleaner-2.22.jar
entry: /Users/vmassol/.m2/repository/org/jdom/jdom2/2.0.6/jdom2-2.0.6.jar
entry: /Users/vmassol/.m2/repository/xerces/xercesImpl/2.12.0/xercesImpl-2.12.0.jar
entry: /Users/vmassol/.m2/repository/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.jar
entry: /Users/vmassol/.m2/repository/net/java/dev/stax-utils/stax-utils/20070216/stax-utils-20070216.jar
entry: /Users/vmassol/.m2/repository/com/fasterxml/woodstox/woodstox-core/6.0.1/woodstox-core-6.0.1.jar
entry: /Users/vmassol/.m2/repository/org/codehaus/woodstox/stax2-api/4.2/stax2-api-4.2.jar
entry: /Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-tool-test-component/11.9-SNAPSHOT/xwiki-commons-tool-test-component-11.9-SNAPSHOT.jar
entry: /Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-tool-test-simple/11.9-SNAPSHOT/xwiki-commons-tool-test-simple-11.9-SNAPSHOT.jar
entry: /Users/vmassol/.m2/repository/junit/junit/4.12/junit-4.12.jar
entry: /Users/vmassol/.m2/repository/org/hamcrest/hamcrest-core/2.1/hamcrest-core-2.1.jar
entry: /Users/vmassol/.m2/repository/org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar
entry: /Users/vmassol/.m2/repository/org/junit/jupiter/junit-jupiter/5.5.2/junit-jupiter-5.5.2.jar
entry: /Users/vmassol/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.5.2/junit-jupiter-api-5.5.2.jar
entry: /Users/vmassol/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar
entry: /Users/vmassol/.m2/repository/org/junit/platform/junit-platform-commons/1.5.2/junit-platform-commons-1.5.2.jar
entry: /Users/vmassol/.m2/repository/org/junit/jupiter/junit-jupiter-params/5.5.2/junit-jupiter-params-5.5.2.jar
entry: /Users/vmassol/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.5.2/junit-jupiter-engine-5.5.2.jar
entry: /Users/vmassol/.m2/repository/org/junit/platform/junit-platform-launcher/1.5.2/junit-platform-launcher-1.5.2.jar
entry: /Users/vmassol/.m2/repository/org/apiguardian/apiguardian-api/1.1.0/apiguardian-api-1.1.0.jar
entry: /Users/vmassol/.m2/repository/org/junit/platform/junit-platform-engine/1.5.2/junit-platform-engine-1.5.2.jar
entry: /Users/vmassol/.m2/repository/org/junit/vintage/junit-vintage-engine/5.5.2/junit-vintage-engine-5.5.2.jar
entry: /Users/vmassol/.m2/repository/org/hamcrest/hamcrest-library/2.1/hamcrest-library-2.1.jar
entry: /Users/vmassol/.m2/repository/org/jmock/jmock/2.6.0/jmock-2.6.0.jar
entry: /Users/vmassol/.m2/repository/org/jmock/jmock-junit4/2.6.0/jmock-junit4-2.6.0.jar
entry: /Users/vmassol/.m2/repository/commons-io/commons-io/2.6/commons-io-2.6.jar
entry: /Users/vmassol/.m2/repository/org/mockito/mockito-core/3.1.0/mockito-core-3.1.0.jar
entry: /Users/vmassol/.m2/repository/net/bytebuddy/byte-buddy/1.10.1/byte-buddy-1.10.1.jar
entry: /Users/vmassol/.m2/repository/net/bytebuddy/byte-buddy-agent/1.9.10/byte-buddy-agent-1.9.10.jar
entry: /Users/vmassol/.m2/repository/org/objenesis/objenesis/2.6/objenesis-2.6.jar
entry: /Users/vmassol/.m2/repository/org/mockito/mockito-junit-jupiter/3.1.0/mockito-junit-jupiter-3.1.0.jar
entry: /Users/vmassol/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar
entry: /Users/vmassol/.m2/repository/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar
entry: /Users/vmassol/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar
entry: /Users/vmassol/.m2/repository/org/slf4j/log4j-over-slf4j/1.7.28/log4j-over-slf4j-1.7.28.jar
entry: /Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-observation-api/11.9-SNAPSHOT/xwiki-commons-observation-api-11.9-SNAPSHOT.jar
entry: /Users/vmassol/.m2/repository/jmock/jmock/1.2.0/jmock-1.2.0.jar
entry: /Users/vmassol/.m2/repository/jmock/jmock-cglib/1.2.0/jmock-cglib-1.2.0.jar
entry: /Users/vmassol/.m2/repository/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.jar
entry: /Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-component-default/11.9-SNAPSHOT/xwiki-commons-component-default-11.9-SNAPSHOT.jar
entry: /Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-component-observation/11.9-SNAPSHOT/xwiki-commons-component-observation-11.9-SNAPSHOT.jar
entry: /Users/vmassol/.m2/repository/org/xwiki/commons/xwiki-commons-configuration-api/11.9-SNAPSHOT/xwiki-commons-configuration-api-11.9-SNAPSHOT.jar
entry: /Users/vmassol/.m2/repository/commons-beanutils/commons-beanutils/1.9.4/commons-beanutils-1.9.4.jar
entry: /Users/vmassol/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar
entry: /Users/vmassol/.m2/repository/org/evosuite/evosuite-standalone-runtime/1.0.6/evosuite-standalone-runtime-1.0.6.jar
[MASTER] 10:13:38.122 [main] ERROR EvoSuite - Fatal crash on main EvoSuite process. Class using seed 1570436018117. Configuration id : null
java.lang.IllegalArgumentException: Classpath element does not exist on disk at: target/test-classses
at org.evosuite.classpath.ClassPathHandler.checkIfValidClasspathEntry(ClassPathHandler.java:212) ~[evosuite-master-1.0.7-SNAPSHOT.jar:1.0.7-SNAPSHOT]
at org.evosuite.classpath.ClassPathHandler.getClassPath(ClassPathHandler.java:119) ~[evosuite-master-1.0.7-SNAPSHOT.jar:1.0.7-SNAPSHOT]
at org.evosuite.classpath.ClassPathHandler.changeTargetClassPath(ClassPathHandler.java:105) ~[evosuite-master-1.0.7-SNAPSHOT.jar:1.0.7-SNAPSHOT]
at org.evosuite.CommandLineParameters.handleClassPath(CommandLineParameters.java:236) ~[evosuite-master-1.0.7-SNAPSHOT.jar:1.0.7-SNAPSHOT]
at org.evosuite.EvoSuite.parseCommandLine(EvoSuite.java:222) ~[evosuite-master-1.0.7-SNAPSHOT.jar:1.0.7-SNAPSHOT]
at org.evosuite.EvoSuite.main(EvoSuite.java:341) ~[evosuite-master-1.0.7-SNAPSHOT.jar:1.0.7-SNAPSHOT]
vmassol ~/dev/xwiki/xwiki-commons/xwiki-commons-core/xwiki-commons-xml (master *+) $ Any idea? Thx!
pderakhshanfar
commented
5 years ago According to the error, the binaries are not available at target/test-classses. I think there is a typo in the projectCP (test-classses).
pderakhshanfar
commented
5 years ago Hi @pderakhshanfar
Thanks. However, as you can see on #89 (comment) I've already used
-Dno_runtime_dependency=trueto generate the tests above they still require some dependency. Is that a bug?Thanks
I think we still have the EvoSuite dependencies in the generated tests because EvoSuite tried to mock some files which are needed by the class under test. If you want to remove that part, you should disable the virtual file system by -Dvirtual_fs=false. However, it will decrease the ability of EvoSuite to test the class. Is it okay for you if you use the EvoSuite library in the test generation process?
vmassol
commented
5 years ago Thanks for spotting it Pouria! And sorry for the stupid question :)
vmassol
commented
5 years ago To summarize, there are 3 outstanding issues ATM logged in this issue:
-Dno_runtime_dependency=true is set-prefix parameter doesn't seem to work (see above). Note that using -target "target/classes" is working though which makes it good to generate tests for a full maven module.vmassol
commented
5 years ago FTR found a new issue for xwiki for which I need to find a solution for. See https://forum.xwiki.org/t/include-tests-generated-by-botsing-evosuite-in-our-build/5648/11?u=vmassol
vmassol
commented
5 years ago
Hi, just rebuilt evosuite/botsing from sources:
git clone https://github.com/STAMP-project/evosuite.gitmvn clean install -DskipTestsgit clone https://github.com/STAMP-project/botsing.gitpom.xmlcd botsing-reproduction/andmvn install:install-file -Dfile=evosuite-client-botsing-1.0.7.jar -DgroupId=org.evosuite -DartifactId=evosuite-client-botsing -Dversion=1.0.7-SNAPSHOT -Dpackaging=jarmvn clean install -DskipTests~/.m2/settings.xml, see https://dev.xwiki.org/xwiki/bin/view/Community/Building/#HInstallingMavengit clone https://github.com/xwiki/xwiki-commons.gitcd xwiki-commons/xwiki-commons-core/xwiki-commons-xmlandmvn clean installmvn dependency:build-classpathand copy results in$xwikicpshell variable:java -d64 -Xmx10000m -jar ~/.m2/repository/eu/stamp-project/botsing-model-generation/1.0.8-SNAPSHOT/botsing-model-generation-1.0.8-SNAPSHOT-jar-with-dependencies.jar -project_cp "$xwikicp" -project_prefix "org.xwiki.xml" -out_dir target/botsingResults:
Note: Tests are written in JUnit5.
There are tests:
EDIT 2019-10-07: See https://forum.xwiki.org/t/include-tests-generated-by-botsing-evosuite-in-our-build/5648/4