search

automl / autoweka

Auto-WEKA
http://www.cs.ubc.ca/labs/beta/Projects/autoweka/
330 stars 105 forks source link

Fix NullPointerException error when launching WEKA with relative path… #96

Closed justinuliu closed 2 years ago

justinuliu commented 2 years ago

… of weka.jar

When launching weka with command such as "java -jar weka.jar", AutoWeka will raise NullPointerException because it cannot find weka.jar.

The problem is that the function getAbsoluteClasspath in Util simply return "java.class.path" property (so the value could be a relative path as we give when launching WEKA). When the value of path is used to run AutoWeka tasks, they will not be able to find weka.jar due to the work directory of those tasks is in the other location (usually in /tmp/autowekaxxxxxxxxxx). I think it is also related to the removed dedicated weka.jar from AutoWeka in the release 2.6.3.

larskotthoff commented 2 years ago

Thanks, but I'm still getting this error the NullPointerException when running from the commandline on Linux. What system did you test this on and how?

justinuliu commented 2 years ago

Thanks, but I'm still getting this error the NullPointerException when running from the commandline on Linux. What system did you test this on and how?

I test it on Ubuntu 20.04 with openjdk 11.0.11. The process is:

  1. Launch WEKA with command "java -jar weka.jar" in weka-3-9-5 folder
  2. Launch Weka Explorer
  3. Click "open file" to choose a dataset file such as iris.arff
  4. Click Auto-WEKA lable
  5. Click Start button

With this patch, Auto-WEKA should be working well otherwise it will raise a NullPointerException.

In the file autoweka.scenario in task folder in /tmp, we should see a variable like:

algo = "/usr/lib/jvm/java-11-openjdk-amd64/bin/java" -Dautoweka.infinity=1.0E100 -Xmx1024m -cp "weka.jar:/home/justinliu/Downloads/weka-3-9-5/weka.jar:/home/justinliu/wekafiles/packages/Auto-WEKA/autoweka.jar" autoweka.smac.SMACWrapper -prop instanceGenerator=autoweka.instancegenerators.CrossValidation:resultMetric=errorRate:datasetString=classIndex=4__COLONESCAPE__:trainArff=/tmp/autoweka17343316941045276102/Auto-WEKA/Auto-WEKA.arff__COLONESCAPE__:type=trainTestArff__COLONESCAPE__:testArff=__dummy__ -prop initialIncumbent=DEFAULT:acq-func=EI -wrapper

With this patch, there should be two paths of weka.jar for classpath (i.e. -cp), one is relative and the other is absolute path.

larskotthoff commented 2 years ago

I've tried what you've described with OpenJDK 11.0.13 and I'm getting a NullPointerException.

justinuliu commented 2 years ago

I've tried what you've described with OpenJDK 11.0.13 and I'm getting a NullPointerException.

Sorry, I haven't noticed that there are two NullPointerExeception related to AutoWeka. The one I want to fix with this PR could cause AutoWeka to be interrupted with release 2.6.3 after we click start. The backtrace looks like:

[Thread-5] INFO weka.classifiers.meta.AutoWEKAClassifier - classifier: null, arguments: [], attribute search: null, at
tribute search arguments: [], attribute evaluation: null, attribute evaluation arguments: []
java.lang.NullPointerException
        java.base/java.lang.String.contains(String.java:2036)
        weka.core.ClassCache.find(ClassCache.java:425)
        weka.core.ClassDiscovery.find(ClassDiscovery.java:212)
        weka.Run.findSchemeMatch(Run.java:82)
        weka.core.ResourceUtils.forName(ResourceUtils.java:76)
        weka.core.Utils.forName(Utils.java:1112)
        weka.classifiers.AbstractClassifier.forName(AbstractClassifier.java:91)
        weka.classifiers.meta.AutoWEKAClassifier.buildClassifier(AutoWEKAClassifier.java:519)
        weka.gui.explorer.AutoWEKAPanel$13.run(AutoWEKAPanel.java:451)

        at java.base/java.lang.String.contains(String.java:2036)
        at weka.core.ClassCache.find(ClassCache.java:425)
        at weka.core.ClassDiscovery.find(ClassDiscovery.java:212)
        at weka.Run.findSchemeMatch(Run.java:82)           
        at weka.core.ResourceUtils.forName(ResourceUtils.java:76) 
        at weka.core.Utils.forName(Utils.java:1112)
        at weka.classifiers.AbstractClassifier.forName(AbstractClassifier.java:91)  
        at weka.classifiers.meta.AutoWEKAClassifier.buildClassifier(AutoWEKAClassifier.java:519)
        at weka.gui.explorer.AutoWEKAPanel$13.run(AutoWEKAPanel.java:451)

The other NullPointerException should occur at loading dataset via "choose file..." button. It seems not affecting the process of AutoWeka. The backtrace is:

        weka.gui.explorer.ClassifierPanel.updateCapabilitiesFilter(ClassifierPanel.java:3471)                                 weka.gui.explorer.ClassifierPanel.capabilitiesFilterChanged(ClassifierPanel.java:3511)
        weka.gui.explorer.Explorer.notifyCapabilitiesFilterListener(Explorer.java:351)                                        weka.gui.explorer.PreprocessPanel$21.run(PreprocessPanel.java:776)
        java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:303)
        java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:770)                   
        java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)                                  
        java.desktop/java.awt.EventQueue$4.run(EventQueue.java:715)                                                           java.base/java.security.AccessController.doPrivileged(Native Method)                
        java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:
85)                                                                                                                   
        java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:740)
        java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
        java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
        java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
        java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
        java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
        java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)           

        at weka.core.Instances.<init>(Instances.java:179)
        at weka.gui.explorer.ClassifierPanel.updateCapabilitiesFilter(ClassifierPanel.java:3471)
        at weka.gui.explorer.ClassifierPanel.capabilitiesFilterChanged(ClassifierPanel.java:3511)
        at weka.gui.explorer.Explorer.notifyCapabilitiesFilterListener(Explorer.java:351)
        at weka.gui.explorer.PreprocessPanel$21.run(PreprocessPanel.java:776)
        at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:303)
        at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:770)
        at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)            
        at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:715)                          
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)                                                                                                                        at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:740)                                                at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)                     at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
        at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
        at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
        at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
        at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)
larskotthoff commented 2 years ago

I'm still getting the first exception when starting to run Auto-WEKA.

justinuliu commented 2 years ago

I'm still getting the first exception when starting to run Auto-WEKA.

Could you take look at the file "autoweka.scenario" in the folder /tmp/autowekaxxxxxxxxxxxx/Auto-WEKA. In the line of variable assignment for "algo", Could you see the absolute path of "weka.jar" for "-cp" option like: -cp "weka.jar:/home/justinliu/Downloads/weka-3-9-5/weka.jar:/home/justinliu/wekafiles/packages/Auto-WEKA/autoweka.jar" ? Or there is only relative path like: -cp "weka.jar:/home/justinliu/wekafiles/packages/Auto-WEKA/autoweka.jar".

larskotthoff commented 2 years ago

There's a relative path.

justinuliu commented 2 years ago

There's a relative path.

I think the PR haven't been applied in your test.

larskotthoff commented 2 years ago

It definitely is. I checked out your fork of the repo and the branch with your modifications. I'm getting a relative path in the autoweka.scenario file for the classpath for weka.jar when running through the GUI. I've doubled checked, rebuilt the package and reinstalled through the package manager, same result.

justinuliu commented 2 years ago

Could you overwrite "autoweka.jar" in the folder wekafiles/packages/Auto-WEKA with the one you rebuilt in the root of Auto-WEKA project. That's what I did to apply this change.

larskotthoff commented 2 years ago

Huh, that worked. Not sure why the package manager didn't do that.

Thanks, merging.