[Request] Provide a permanent link to the latest stable release

[PGijsbers]

I realize this is perhaps a bit moot since the project is not actively worked on anymore, but would it be possible to host the latest release of AutoWEKA under a permanent URL?

With the AutoML benchmark we provide users the option to run the latest stable release of any integrated framework. Currently there is no link (as far as we are aware) that always refers to the latest stable release of Auto-WEKA, e.g. https://www.cs.ubc.ca/labs/beta/Projects/autoweka/autoweka-latest.zip.

Are new releases (e.g. with the cleanup commit from a few weeks ago) uploaded under the link to 2.6? If so we can technically also use that as a "latest", though if a 2.7 ever does release we wouldn't automatically pick that up.

[larskotthoff]

Sure, I've added the latest version to the repo. Permanent URL: https://raw.githubusercontent.com/automl/autoweka/master/autoweka-latest.zip

[PGijsbers]

Thanks a lot!

[PGijsbers]

Should this jar be used differently than auto-weka 2.6? I get an error message that the class can't be found. Using Auto-WEKA latest:

root@831489b45d81:/bench# wget https://raw.githubusercontent.com/automl/autoweka/master/autoweka-latest.zip
...
2021-06-30 11:38:38 (40.2 MB/s) - ‘autoweka-latest.zip’ saved [20075432/20075432]
root@831489b45d81:/bench# unzip autoweka-latest.zip -d latest
root@831489b45d81:/bench# java -cp latest/autoweka.jar weka.classifiers.meta.AutoWEKAClassifier
Error: Could not find or load main class weka.classifiers.meta.AutoWEKAClassifier

Using Auto-WEKA 2.6 it behaves as expected:

root@831489b45d81:/bench# wget http://www.cs.ubc.ca/labs/beta/Projects/autoweka/autoweka-2.6.zip
...
2021-06-30 11:42:20 (7.95 MB/s) - ‘autoweka-2.6.zip’ saved [29846003/29846003]
root@831489b45d81:/bench# unzip autoweka-2.6.zip -d old
root@831489b45d81:/bench# java -cp old/autoweka.jar weka.classifiers.meta.AutoWEKAClassifier

Weka exception: No training file and no object input file given.

[larskotthoff]

Argh, looks like this is a side effect of the latest changes that don't include the weka.jar anymore. You'll have to include this in the classpath when running this way now:

java -cp latest/autoweka.jar:/somewhere/weka.jar weka.classifiers.meta.AutoWEKAClassifier

Let me know if this works for you.

[PGijsbers]

Thanks, works like a charm! Two quick questions, if you don't mind:

1. Is the exclusion of the weka.jar file from AutoWEKA downloads permanent, or should we expect it to return?
2. Is there a way to check the version of AutoWEKA? I don't see anything in the manual, and the MANIFEST.MF does not contain an Implementation-Version field.

[larskotthoff]

1. This was done in a PR by the WEKA folks, and I can see the reasoning behind it, so expect it to be permanent. The lib/ directory still contains the weka.jar that used to be included though, so you could simply use that.
2. There is not, but you can work around this by always including weka.jar on the classpath.

[PGijsbers]

There is not, but you can work around this by always including weka.jar on the classpath.

Sorry if I misunderstood, but it sounds like that would refer to the WEKA version? Is that not independent from the AutoWEKA version?

[larskotthoff]

Yes -- I interpreted your question as using the Auto-WEKA version to determine whether you need to include the weka.jar, which you can avoid as outlined above.

[PGijsbers]

No, this was to be able to record which version of Auto-WEKA is used when a user runs an experiment with the automl benchmark software. The benchmark tool automatically records the version of the AutoML framework used. This is useful for reproducibility (especially paired with automatically being able to download those older versions to run experiments).

[larskotthoff]

Hmm, could you record the checksum of the release file?

[PGijsbers]

We'll resort to that to at least record a version. It still is hard to then (automatically) install specific older versions. E.g. ML-Plan hosts older downloads under https://download.mlplan.org/version/<VERSION>, which means it's easy to reproduce results even if they release new versions down the road (or to compare different versions). Github could work going forward (even if we shouldn't really expect any new releases).

[larskotthoff]

Yeah, this will unfortunately not work at all for older releases, and I don't have control over the old location anymore (including not being able to guarantee that the old files will remain there).

[PGijsbers]

Good to know, thanks. 👍