Open edwardmp opened 8 years ago
I don't know this code, nor do I understand Eclipse's AST data structures. @juliandolby any thoughts?
@edwardmp do you see the same stack as in https://sourceforge.net/p/wala/mailman/message/34746738/ ?
Thanks for the quick response. The stack changes a bit depending on which program I try to analyze, but resolveMethodBinding is the method being called which causes the nullpointerexception, all in invarious visitor methods in that class.
Example trace:
Exception in thread "main" java.lang.NullPointerException
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:1480)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitNode(JDTJava2CAstTranslator.java:2901)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:1462)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitNode(JDTJava2CAstTranslator.java:2887)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitNodeOrNodes(JDTJava2CAstTranslator.java:2959)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.createBlock(JDTJava2CAstTranslator.java:1275)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:1280)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitNode(JDTJava2CAstTranslator.java:2861)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:806)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.createClassDeclaration(JDTJava2CAstTranslator.java:485)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitTypeDecl(JDTJava2CAstTranslator.java:408)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:2828)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.translateToCAst(JDTJava2CAstTranslator.java:261)
at com.ibm.wala.cast.java.translator.jdt.ejc.EJCSourceModuleTranslator$EjcAstToIR.acceptAST(EJCSourceModuleTranslator.java:92)
at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:1029)
at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:636)
at org.eclipse.jdt.core.dom.ASTParser.createASTs(ASTParser.java:990)
at com.ibm.wala.cast.java.translator.jdt.ejc.EJCSourceModuleTranslator.loadAllSources(EJCSourceModuleTranslator.java:181)
at com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl.loadAllSources(JavaSourceLoaderImpl.java:505)
at com.ibm.wala.classLoader.ClassLoaderImpl.init(ClassLoaderImpl.java:529)
at com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl.init(JavaSourceLoaderImpl.java:516)
at com.ibm.wala.cast.java.translator.jdt.JDTClassLoaderFactory.makeNewClassLoader(JDTClassLoaderFactory.java:69)
at com.ibm.wala.classLoader.ClassLoaderFactoryImpl.getLoader(ClassLoaderFactoryImpl.java:69)
at com.ibm.wala.ipa.cha.ClassHierarchy.<init>(ClassHierarchy.java:231)
at com.ibm.wala.ipa.cha.ClassHierarchy.<init>(ClassHierarchy.java:177)
at com.ibm.wala.ipa.cha.ClassHierarchy.make(ClassHierarchy.java:1230)
at com.ibm.wala.cast.java.client.JavaSourceAnalysisEngine.buildClassHierarchy(JavaSourceAnalysisEngine.java:135)
at eu.sig.aevas.callgraph.WalaStatic.run(WalaStatic.java:165)
at eu.sig.aevas.AeVas.main(AeVas.java:16)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Other input program stack trace:
Exception in thread "main" java.lang.NullPointerException
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator$TryCatchContext.<init>(JDTJava2CAstTranslator.java:3150)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:2684)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitNode(JDTJava2CAstTranslator.java:2939)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitNodeOrNodes(JDTJava2CAstTranslator.java:2959)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.createBlock(JDTJava2CAstTranslator.java:1275)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:1280)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitNode(JDTJava2CAstTranslator.java:2861)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:806)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.createClassDeclaration(JDTJava2CAstTranslator.java:485)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitTypeDecl(JDTJava2CAstTranslator.java:408)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:2828)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.translateToCAst(JDTJava2CAstTranslator.java:261)
at com.ibm.wala.cast.java.translator.jdt.ejc.EJCSourceModuleTranslator$EjcAstToIR.acceptAST(EJCSourceModuleTranslator.java:92)
at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:1029)
at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:636)
at org.eclipse.jdt.core.dom.ASTParser.createASTs(ASTParser.java:990)
at com.ibm.wala.cast.java.translator.jdt.ejc.EJCSourceModuleTranslator.loadAllSources(EJCSourceModuleTranslator.java:181)
at com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl.loadAllSources(JavaSourceLoaderImpl.java:505)
at com.ibm.wala.classLoader.ClassLoaderImpl.init(ClassLoaderImpl.java:529)
at com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl.init(JavaSourceLoaderImpl.java:516)
at com.ibm.wala.cast.java.translator.jdt.JDTClassLoaderFactory.makeNewClassLoader(JDTClassLoaderFactory.java:69)
at com.ibm.wala.classLoader.ClassLoaderFactoryImpl.getLoader(ClassLoaderFactoryImpl.java:69)
at com.ibm.wala.ipa.cha.ClassHierarchy.<init>(ClassHierarchy.java:231)
at com.ibm.wala.ipa.cha.ClassHierarchy.<init>(ClassHierarchy.java:177)
at com.ibm.wala.ipa.cha.ClassHierarchy.make(ClassHierarchy.java:1230)
at com.ibm.wala.cast.java.client.JavaSourceAnalysisEngine.buildClassHierarchy(JavaSourceAnalysisEngine.java:135)
at eu.sig.aevas.callgraph.WalaStatic.run(WalaStatic.java:165)
at eu.sig.aevas.AeVas.main(AeVas.java:16)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
By the way I use 1.3.9-SNAPSHOT versions of all WALA libraries, since I could not install some through maven (com.ibm.wala.cast.java and com.ibm.wala.ide.jdt)
I think I know why the issue arose:
I had to add jars of all libraries to by calling addSystemModule of class JavaSourceAnalysisEngine.
I needed to add rt.jar
when my source files used no external libraries, and if they did .jars with all external libraries.
Thanks for digging in! We still shouldn't crash for missing jars. We should add some catch blocks that log a warning.
Yes I agree that a more helpful message would be convenient.
Also @msridhar, is there any way com.ibm.wala.cast.java
and com.ibm.wala.ide.jdt
could be made available in the maven repository?
Finally, if I add my source files through addSourceModule why doesn't the condition is this codeblock ever to false? Any clue?
if (!analysisScope.isApplicationLoader(c.getClassLoader())) {
continue;
}
I see it is a source loader and not an application loader. It works if I try to get the parent of the source loader, which is a application loader. But isn't there a better way?
@msridhar ideally, in my case, I would like to continue building the dependency graph when a lib jar is missing, just behave as if that library would have been added to the exclusions file. Is this possible?
@edwardmp that's what I was suggesting above. A hackish fix would be to add null checks and then just soldier on without the relevant method / field. WALA is generally robust to missing classes, etc., it's just this frontend that is falling down. I am very busy for the next few days, but I can try to take a look after that. Or, if you could attempt to throw together a patch and make a pull request, I can give comments.
Re: Maven Central, the problem is that these projects depend on Eclipse, and I'm not sure of the right way to specify such dependencies in a Maven jar. There must be some way to do it (after all, we can build WALA with Maven), I just don't know how to do it. Again, if someone could do some legwork here that would be helpful.
Re: addSourceModule
, I don't know the answer. @juliandolby any thoughts?
@msridhar OK. Regarding the availability in the Maven repository, I am trying to add the missing projects but I am not that experienced with Maven and the setup is a bit complex, will definitely dive deeper into this later.
In the meantime, I want to use WALA in a production environment at the company I do an internship at, but they refuse to let me use the current 1.3.9-SNAPSHOT. Is there any way you could release 1.3.9 anytime soon?
Thanks in advance,
I'm fine with cutting a new release. Let me check with Julian. --Manu
On Mon, May 9, 2016 at 9:41 PM Edward Poot notifications@github.com wrote:
@msridhar https://github.com/msridhar OK. Regarding the availability in the Maven repository, I am trying to add the missing projects but I am not that experienced with Maven and the setup is a bit complex, will definitely dive deeper into this later.
In the meantime, I want to use WALA in a production environment at the company I do an internship at, but they refuse to let me use the current 1.3.9-SNAPSHOT. Is there any way you could release 1.3.9 anytime soon?
Thanks in advance,
— You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub https://github.com/wala/WALA/issues/99#issuecomment-217853334
I have tagged release 1.3.9 in the git repository (R_1.3.9
). Still working on pushing new jars to Maven Central, should be done in the next day or so.
Great, thanks a lot.
Hi @msridhar,
Did you have any opportunity yet to change how missing library jars are handled? This would be a great case for the use of Optionals in Java 8, although personally I have no experience in using them.
Haven't had time to look at this yet, and not sure when I will. At the least, anyone trying to fix will need a test case for reproducing the error, along with instructions on how to reproduce.
On Wed, May 18, 2016, 10:28 AM Edward Poot notifications@github.com wrote:
Hi @msridhar https://github.com/msridhar,
Did you have any opportunity yet to change how missing library jars are handled? This would be a great case for the use of Optionals in Java 8, although personally I have no experience in using them.
— You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub https://github.com/wala/WALA/issues/99#issuecomment-220099734
OK. I can certainly temporary fix it, although doing it in a proper way requires much knowhow of how the library operates. Tomorrow I will put some example code in a repository so it can be easily reproduced.
Hi @msridhar
Finally had some time to provide an example displaying the issue. This is the tiny system I want to create a callgraph for: https://github.com/edwardmp/wala-issue-example This tiny example depends on the Netty library.
Now I want to only use the source files to generate the callpgrah, using the Eclipse Java frontend.
This is the code used to generate the callgraph: https://github.com/edwardmp/wala-callgraph-generation-issue-example
The code in main
first tries to generate the graph without adding the Netty.jar as a system module. Run it and you will see the null pointer exception. The commented out code in main
shows that if you do run it and specify the jar, everything works OK.
Ideally, WALA should log an error but not throw an exception and keep generating the call graph but just exclude calls to that library.
Don't forget to change the hardcoded path in main
to point to the the download location of the first repository I mentioned (hardcoded path in my example is /Users/Edward/Desktop/wala-issue-example
).
Hope this helps!
@msridhar is it OK if a open a pull request in which some null checks are added?
Sure, that would be greatly appreciated! By the way, as you may have
noticed, I've done some refactoring on the master branch. From now on, you
should be able to use the wala.cast.java
and wala.cast.java.ecj
projects using Maven Central jars. If you'd like to test it out, I can
create some snapshot jars; let me know. If this is what you needed, we can
cut a new release for it.
--Manu
On Mon, May 30, 2016 at 12:56 AM Edward Poot notifications@github.com wrote:
@msridhar https://github.com/msridhar is it OK if a open a pull request in which some null checks are added?
— You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/wala/WALA/issues/99#issuecomment-222434503, or mute the thread https://github.com/notifications/unsubscribe/AALyUSZMLwgxCnwR1RpvMDMBo_9vVb-aks5qGpgZgaJpZM4IS85m .
@msridhar That is very convenient, but what is more important is the problem mentioned above leading to a NPE.
I've been pulling my hair out the last few hours trying to run WALA on for example PMD (https://github.com/pmd/pmd, ran on pmd-core dir), but this again leads to a null pointer exception. But in this case, the reason doesn't seem to be related to missing jars from dependencies.
Caused by: java.lang.NullPointerException
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.createClassDeclaration(JDTJava2CAstTranslator.java:473)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visitTypeDecl(JDTJava2CAstTranslator.java:408)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.visit(JDTJava2CAstTranslator.java:2828)
at com.ibm.wala.cast.java.translator.jdt.JDTJava2CAstTranslator.translateToCAst(JDTJava2CAstTranslator.java:261)
at com.ibm.wala.cast.java.translator.jdt.ejc.EJCSourceModuleTranslator$EjcAstToIR.acceptAST(EJCSourceModuleTranslator.java:92)
at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:1029)
at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:636)
at org.eclipse.jdt.core.dom.ASTParser.createASTs(ASTParser.java:990)
at com.ibm.wala.cast.java.translator.jdt.ejc.EJCSourceModuleTranslator.loadAllSources(EJCSourceModuleTranslator.java:181)
at com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl.loadAllSources(JavaSourceLoaderImpl.java:505)
at com.ibm.wala.classLoader.ClassLoaderImpl.init(ClassLoaderImpl.java:529)
at com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl.init(JavaSourceLoaderImpl.java:516)
at com.ibm.wala.cast.java.translator.jdt.JDTClassLoaderFactory.makeNewClassLoader(JDTClassLoaderFactory.java:69)
at com.ibm.wala.classLoader.ClassLoaderFactoryImpl.getLoader(ClassLoaderFactoryImpl.java:69)
at com.ibm.wala.ipa.cha.ClassHierarchy.<init>(ClassHierarchy.java:231)
at com.ibm.wala.ipa.cha.ClassHierarchy.<init>(ClassHierarchy.java:177)
at com.ibm.wala.ipa.cha.ClassHierarchy.make(ClassHierarchy.java:1230)
at com.ibm.wala.cast.java.client.JavaSourceAnalysisEngine.buildClassHierarchy(JavaSourceAnalysisEngine.java:135)
This is caused because IVariableBinding fieldBinding = fieldFrag.resolveBinding();
has a value of null, thus it is again related to the resolving of bindings. It could be that the issue I opened this issue for isn't the real issue but just a symptom. Something is seriously wrong in resolving bindings.
Kind of frustrating as I really need to generate a call graph of various systems for research related to my thesis and I don't have much time to dive into this.
@edwardmp do you really need to use the Java source front end? All the problems here are in the source front end. The Java bytecode front end is much more robust.
@msridhar Unfortunately, the company I conduct my research at only has certain clients' systems source code available and (usually) not any binaries. I tried a few other call graph libraries before using WALA but those wouldn't work either.
Ok. What makes you think this is not due to missing jars? In any case, as a patch-up fix, my approach would be to add null checks around any relevant calls to resolveBindings()
, and then "bail out" if the reset is null. E.g., for the stack you just posted, we just wouldn't add the FieldEntity
on the next line. If you submit a PR that adds null checks of this form, I'll take a quick look at it and then merge.
@msridhar Well basically because I have checked that I have jars of all dependencies. Also, in the PMD case I deleted most source files and traced it to 1 specific file (that only had a dependency on a subdir).
OK I will try to add some null checks. With bailing out you mean just skipping the next line or returning from the method? Thanks for your cooperation! much appreciated
By bailing out I generally mean for whatever operation the binding is being used for (typically adding something to a class or class hierarchy), don't do that operation. But if you're a little over-aggressive in bailing out, that's fine for now (better than crashing).
@msridhar spent the last few hours adding checks but this is not really a solution; sometimes you can skip a part of a method but other times you can only return null from that method in case the binding is null, which just propagates the NPE to another place.. Digging in to find the real reason the bindings are not set but no luck so far
@msridhar
So far it boils down to this: setRecoveryBindings must be set to true when the ASTParser is created.
I have a small test case where a ClassInstanceCreation can not resolve the constructor bindings while all other bindings are actually present.
Example statement: new ArrayList<>(10);
does have constructor bindings and new ArrayList<>();
(without a capacity) does have bindings . But when the capacity is dependent upon another list (e.g. new ArrayList<>(otherList.size());
it does not have bindings` Why is still the big question..
@msridhar Here is a more complete example:
public abstract class AbstractPropertySource implements PropertySource {
protected List<PropertyDescriptor<?>> copyPropertyDescriptors() {
List<PropertyDescriptor<?>> b = new ArrayList<>(); // works, resolveConstructorBindings() does not return null
List<PropertyDescriptor<?>> c = new ArrayList<>(1); // works fine
List<PropertyDescriptor<?>> copy = new ArrayList<>(b.size()); // null pointer exception because resolveConstructorBinding() returns null
copy.addAll(propertyDescriptors);
return copy;
}
}
Maybe its a bug in the JDT library?
Thanks for digging in! So is adding a call to setRecoveryBindings somewhere sufficient to fix this?
On Tue, May 31, 2016, 11:17 AM Edward Poot notifications@github.com wrote:
@msridhar https://github.com/msridhar Here is a more complete example:
public abstract class AbstractPropertySource implements PropertySource { protected List<PropertyDescriptor<?>> copyPropertyDescriptors() { List<PropertyDescriptor<?>> b = new ArrayList<>(); // works, v() does not return null List<PropertyDescriptor<?>> copy = new ArrayList<>(b.size()); // null pointer exception because resolveConstructorBinding() returns null copy.addAll(propertyDescriptors); return copy; } }
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wala/WALA/issues/99#issuecomment-222774400, or mute the thread https://github.com/notifications/unsubscribe/AALyUeX46Vhb3eAui2Z3JGjwJxeFgJM-ks5qHHtDgaJpZM4IS85m .
@msridhar
In the code I used to reproduce it at some point it had to setRecoveryBindings to true, but in the ultimate code I have changing it for this example does not have immediate impact.
So no, unfortunately that's not enough. In the example above there are cases where resolveConstructorBindings() still returns null while it should. Trying if I can add some null checks for that, but bailing out there will probably cause issues somewhere else..
Most likely, its a bug in the JDT core. Last version 3.10 was released in early 2015, seems the project is inactive maybe?
I'm going to try filing a bug for JDT. Odd thing is that the bug reporter is listing versions different than 3.10, seems as if JDT core in the bugtracker refers to another project? Any idea?
I have to say this is driving me nuts, as I don't expect a fix any time soon, which means I can't really use WALA for my research. After having setup 3 previous call graph libraries that didn't work this is kind of frustrating.
@edwardmp I'm really sorry this doesn't seem to be working out. I am a bit unsure about the version numbering of JDT Core. I thought we are pulling in the latest version, but I'm not sure. Let me look into it.
Thanks, of course I don't blame you or any other WALA maintainer. It's just annoying it eventually worked fine for a sample project but not any other it seems.
I've been reading up on alternatives for call graph generation. Seems you can use some JDT stuff for that, using the CallHierarchy. Its already in the WALA code somewhere: https://github.com/wala/WALA/blob/e87b2e5d843796246a856be65e7cb7b8251c4a47/com.ibm.wala.ide.jsdt/source/com/ibm/wala/ide/util/JsdtUtil.java
Do you happen to know more about this? Maybe its an alternative approach. See also http://www.programcreek.com/2011/07/find-all-callers-of-a-method/
Yeah, so Eclipse has had this call hierarchy feature for finding callers of a method for a while. I'm guessing you would have to be inside a running Eclipse instance to use this feature, i.e., it'll be hard to use just from the command line. But I have no personal experience trying to programmatically invoke the call hierarchy. ( @juliandolby wrote that Jsdt code; he might be able to chime in with more tips.)
@msridhar
Thanks. Kind of confused as if WALA actually uses the latest JDK core version? The artifact id is different than the one used though: https://github.com/eclipse/eclipse.jdt.core/blob/master/pom.xml
Yeah, it's a bit of a mess. We have the Eclipse plugin build system which uses Tycho, and will bring in some version of JDT Core; I think that's Eclipse Luna, which is not the latest. Then, I've been writing these mvncentral.xml
files which try to make it easier to use WALA outside of Eclipse, and that's what is pulling in 3.10. Is it possible maybe for you to plug in a more recent version and see if that fixes things? If so, we can try to figure out how to update what WALA pulls in.
Yes thats my intention. It is not unthinkable that if its a bug, it may be fixed in newer versions.
Its actually hard to find a JAR with a newer release but I found org.eclipse.jdt.core.source_3.11.2.v20160128-0629.jar
in https://www.eclipse.org/downloads/download.php?file=/eclipse/downloads/drops4/R-4.5.2-201602121500/org.eclipse.jdt-4.5.2.zip (under plugins folder) which seems to contain the necessary files.. experimenting if it might be compatible out of the box.
Ok, let us know how it goes.
@msridhar
OK using so org.eclipse.jdt.core_3.11.2 which is slightly newer still leads to an exception, but it does progress a bit further. Furthermore the exception is not a NPE but Assertions.UNREACHABLE("Couldn't find default ctor");
I tried another call graph library which seemed to work, only to discover that it has issues with handling java 8 source code, so I'm pretty desperate for this to work.
Nope unfortunately, in the end throws NPE at me again. I think I have to reconsider my research subject since every call graph library I tried had issues some way or another.
@edwardmp in cases where you have all the dependent libraries, could you just compile the source code? Then you could avoid these JDT issues.
@msridhar If I do it manually, I might be able for some systems but my employer generally does not necessarily have all (e.g. proprietary) dependencies. But the input for my system using WALA is supposed to be dynamic, e.g. when my employer obtains a new system it should just work without manually compiling it.
@edwardmp ok, unfortunately, I don't know of any system that can handle your needs out of the box, unfortunately. Beyond the JDT issues we've been dealing with, you mentioned Java 8 source code above. While JDT supports Java 8, I don't think the WALA source front end supports constructs like lambdas. So that's another issue.
For right now, the best thing I can think of for using WALA would be to stick a try-catch block inside ECJSourceModuleTranslator.acceptAST()
:
Could you just catch any NPE thrown by the translateToCAst()
call and immediately return? This might not work if translateToCAst()
mutates data structures and leaves things in an inconsistent state. But it's the best thing I can think of at this point. Otherwise, unfortunately WALA may just not work for this purpose.
@msridhar Thanks, I'll try that as a last resort. Actually this library: https://github.com/Sable/soot does work with source code but only not Java 8 code.. "funny" enough.
Lambda's appeared to work in one small sample project.
Well, didn't expect much but catching Null Pointer exceptions works in the short term (but get a lot of "unexpected problems" reported but eventually another exception is thrown. Do note that all missing classes are excluded in the exclusions file.
505. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Lsun/misc/SignalHandler
506. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Lsun/reflect/LangReflectAccess
507. [SEVERE] class com.ibm.wala.classLoader.BytecodeClass$ClassNotFoundWarning : Lsun/security/krb5/JavaxSecurityAuthKerberosAccess
at com.ibm.wala.ipa.callgraph.propagation.PropagationCallGraphBuilder.makeCallGraph(PropagationCallGraphBuilder.java:263)
at com.ibm.wala.client.AbstractAnalysisEngine.buildCallGraph(AbstractAnalysisEngine.java:137)
at com.ibm.wala.client.AbstractAnalysisEngine.defaultCallGraphBuilder(AbstractAnalysisEngine.java:309)
In my case, I'm thinking about an alternative. I can use JDT to visit all MethodInvocations. Then I can find method calls to a specific library I'm interested in. Then I can use WALA to trace calls within this library (for those libraries I do have the jars so running on bytecode shouldn't pose problems).
@msridhar I seems my solution works well. I don't really care about the source code other than that I want to find calls to a specific library. Using ASTParser I obtain those, then I run wala on the library jar.
Great. I do think however that some warnings should be put in place to warn people that the source translation is "experimental/buggy"
Ok, glad you figured something out. We should indeed add a warning in the documentation. I'll leave this issue open until I do so.
On Thu, Jun 2, 2016, 9:44 AM Edward Poot notifications@github.com wrote:
@msridhar https://github.com/msridhar I seems my solution works well. I don't really care about the source code other than that I want to find calls to a specific library. Using ASTParser I obtain those, then I run wala on the library jar.
Great. I do think however that some warnings should be put in place to warn people that the source translation is "experimental/buggy"
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wala/WALA/issues/99#issuecomment-223350590, or mute the thread https://github.com/notifications/unsubscribe/AALyURNBufgE_vS27CKu-c2SioSPRKqjks5qHwh6gaJpZM4IS85m .
@msridhar For one system I encountered the following exception:
[ERROR][2016-06-06 13:23:43][WalaStaticJarInput] Caught exception during callgraph generation: Could not create a entrypoint callsites: 1. [Moderate] class com.ibm.wala.ipa.cha.ClassHierarchy$ClassExclusion : <Application,Lorg/eclipse/jetty/websocket/client/ClientUpgradeRequest> No superclass found for <Application,Lorg/eclipse/jetty/websocket/client/ClientUpgradeRequest> Superclass name Lorg/eclipse/jetty/websocket/api/UpgradeRequest
2. [Moderate] class com.ibm.wala.ipa.cha.ClassHierarchy$ClassExclusion : <Application,Lorg/eclipse/jetty/websocket/client/ClientUpgradeResponse> No superclass found for <Application,Lorg/eclipse/jetty/websocket/client/ClientUpgradeResponse> Superclass name Lorg/eclipse/jetty/websocket/api/UpgradeResponse
3. [Moderate] class com.ibm.wala.ipa.cha.ClassHierarchy$ClassExclusion : <Application,Lorg/eclipse/jetty/websocket/client/WebSocketClient> No superclass found for <Application,Lorg/eclipse/jetty/websocket/client/WebSocketClient> Superclass name Lorg/eclipse/jetty/util/component/ContainerLifeCycle
I noticed this happens when you don't add any entrypoints. But since I check for that before I generate a call graph, in this case that's not the issue. Any idea?
@edwardmp no idea. If it's unrelated to the JDT issue, best to discuss on the mailing list or a separate issue if it's a bug.
Hello,
I'm trying to create a call graph but encounter a null pointer exception. Depending on the java source I use as input the exception is thrown from different visitors, but it always involves "n.resolveMethodBinding()".
resolveMethodBinding returns null because it uses the class BindingResolver's implementation (org.eclipse.jdt.core.dom package) (line 194):
It is likely that DefaultBindingResolver should be used instead of BindingResolver's which features an empty implementation for each method.
The exact same issue was described by someone else here: https://sourceforge.net/p/wala/mailman/message/34746738/
Hope you have any clue how to solve this issue, as I'm using this tool for research done with regard to my thesis.
Best regards, Edward