search

martin-fleck / cra-ttc2016

The Class Responsibility Assignment Case for the Transformation Tool Contest 2016
3 stars 2 forks source link

K. Lano evaluation of Excel #26

Open klano opened 8 years ago

klano commented 8 years ago

This has clearly produced good results, but it is not clear how it uses transformation technologies, instead of existing non-MT optimisation tools.

In addition, the heuristics in Section 3 seem to require tuning of the algorithm to specific CRA problems. The idea of creating a main class could lead to unbalanced class diagrams. Such diagrams are local but not global optima. The approach does not seem to produce optimal solutions (eg., for C, D, E cases), although the efficiency is very impressive.

I was unable to run the CRAIndexCalculator on the generated xmi files as there is some data error in the files.

maximiliano-vela commented 8 years ago

Hi Kevin - that's weird, just tested fresh outputs on the calculator as well as the ones uploaded to our github and they are picked up just fine.

klano commented 8 years ago

We got errors like this:

EXCEL: C:\Users\localadmin\Desktop\TTC2016\Excel_solution>java -jar CRAIndexCalculator. jar TTC_InputRDG_B_output.xmi Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoa der.java:58) Caused by: org.eclipse.emf.ecore.resource.impl.ResourceSetImpl$1DiagnosticWrappe dException: org.xml.sax.SAXParseExceptionpublicId: TTC_InputRDG_B_output.xmi; sy stemId: file:///C:/Users/localadmin/Desktop/TTC2016/Excel_solution/TTCInputRDG B_output.xmi; lineNumber: 52; columnNumber: 71; Attribute name "data-pjax-transi ent" associated with an element type "meta" must be followed by the ' = ' charac ter. at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.handleDemandLoadE xception(ResourceSetImpl.java:319) at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.demandLoadHelper( ResourceSetImpl.java:278) at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.getResource(Resou rceSetImpl.java:406) at at.ac.tuwien.big.momot.examples.modularization.cra.eval.CRAIndexCalcu lator.loadModel(CRAIndexCalculator.java:28) at at.ac.tuwien.big.momot.examples.modularization.cra.eval.CRAIndexCalcu lator.main(CRAIndexCalculator.java:93) ... 5 more Caused by: org.xml.sax.SAXParseExceptionpublicId: TTC_InputRDG_B_output.xmi; sys temId: file:///C:/Users/localadmin/Desktop/TTC2016/Excel_solution/TTC_InputRDG_B _output.xmi; lineNumber: 52; columnNumber: 71; Attribute name "data-pjax-transie nt" associated with an element type "meta" must be followed by the ' = ' charact er. at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAX ParseException(ErrorHandlerWrapper.java:198) at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalErro r(ErrorHandlerWrapper.java:177) at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError( XMLErrorReporter.java:441) at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError( XMLErrorReporter.java:368) at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(X MLScanner.java:1436) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImp l.scanAttribute(XMLDocumentFragmentScannerImpl.java:1537) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImp l.scanStartElement(XMLDocumentFragmentScannerImpl.java:1326) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImp l$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2778) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(X MLDocumentScannerImpl.java:606) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImp l.scanDocument(XMLDocumentFragmentScannerImpl.java:510) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(X ML11Configuration.java:848) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(X ML11Configuration.java:777) at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser. java:141) at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Ab stractSAXParser.java:1213) at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.p arse(SAXParserImpl.java:649) at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParser Impl.java:333) at org.eclipse.emf.ecore.xmi.impl.XMLLoadImpl.load(XMLLoadImpl.java:175)

    at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doLoad(XMLResourceImpl

.java:261) at org.eclipse.emf.ecore.resource.impl.ResourceImpl.load(ResourceImpl.ja va:1518) at org.eclipse.emf.ecore.resource.impl.ResourceImpl.load(ResourceImpl.ja va:1297) at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.demandLoad(Resour ceSetImpl.java:259) at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.demandLoadHelper( ResourceSetImpl.java:274) ... 8 more

Date: Thu, 26 May 2016 14:44:59 -0700 From: notifications@github.com To: cra-ttc2016@noreply.github.com CC: klano@hotmail.com; author@noreply.github.com Subject: Re: [martin-fleck/cra-ttc2016] K. Lano evaluation of Excel (#26)

Hi Kevin - that's weird, just tested fresh outputs on the calculator as well as the ones uploaded to our github and they are picked up just fine.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub

dstrueber commented 8 years ago

Based on the error message, I believe you downloaded and checked the GitHub HTML pages rather than the XMI files themselves.

To get the actual XMI files, you will have to click the "Raw" button, for example on https://github.com/maximiliano-vela/ttc-2016/blob/master/Solution%20Output%20-%20.xmi/TTC_InputRDG_A_output.xmi

You can follow the same instructions to get the Henshin XMIs intended for optimality measurement: https://bitbucket.org/ttc16/ttc16/src/f46b0baf5b63322ecb51a358aaa922a131aaddb1/ttc16-solution/results/best_of/?at=master

klano commented 8 years ago

Yes, now we have successfully run the CRA calculator on the files, and the CRA values are very good - but I noticed that there are only 2 or 3 classes in result models C, D, E.

maximiliano-vela commented 8 years ago

Hi Kevin - Glad everything is working fine now.

In the last few versions of the tool we generated several balanced classes by analyzing the amount of similarities between methods exclusively. Although these diagrams looked really well and the strategy seemed appropiate, we obtained much better results by "boosting" the affinity index for the most related method in the matrix. As a result we obtain a main class containing the majority of the features while keeping the overall strongest relationships in separate classes (in order to favor the cohesion of the diagram).

As the input size grows, more features will naturally be contained in the main class but the CRA index remains stable across all inputs.

Thanks Daniel for the good catch.

Maxi.