search

openml / openml-weka

The OpenmlWeka package
4 stars 3 forks source link

No performance information #3

Closed mcachada closed 7 years ago

mcachada commented 7 years ago

Hi,

I've been performing some tests using OpenmlWeka in both GUI and command line. However, I'm getting "0 Evaluation measures" for my experiments. For example: https://www.openml.org/r/2160355.

Can you please help?

Call information and console output for the run mentioned above:

C:\Windows\system32>java` -Xmx8192m org.openml.weka.experiment.TaskBasedExperiment -T 1 -C weka.classifiers.lazy.IBk -- -K 5 -W 0 -A "weka.core.neighboursearch.LinearNNSearch -A \"weka.core.EuclideanDistance -R first-last\"" [16-05-2017 17:12:21] [OK] [Cache] Obtained from cache: task/task_1.xml Initializing... CrossValidationResultProducer: setting additional measures for split evaluator Iterating... [16-05-2017 17:12:21] [OK] [Cache] Obtained from cache: datadescription/datadescription_1.xml [16-05-2017 17:12:21] [OK] [Cache] Obtained from cache: dataset/dataset_1.ARFF [16-05-2017 17:12:21] [OK] [Cache] Obtained from cache: splits/splits_1.arff class org.openml.weka.experiment.OpenmlClassificationSplitEvaluator [16-05-2017 17:12:24] [OK] [Attribtes] Attributes available: [family, product-type, steel, carbon, hardness, temper_rolling, condition, formability, strength, non-ageing, surface-finish, surface-quality, enamelability, bc, bf, bt, bw%2Fme, bl, m, chrom, phos, cbond, marvi, exptl, ferro, corr, blue%2Fbright%2Fvarn%2Fclean, lustre, jurofm, s, p, shape, thick, width, len, oil, bore, packing, class] [16-05-2017 17:12:24] [OK] [Class] Class attribute: @attribute class {1,2,3,4,5,U} [16-05-2017 17:12:24] [INFO] [Splits] Obtaining folds for Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0 [16-05-2017 17:12:24] [INFO] [Splits] Folds obtained [16-05-2017 17:12:24] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 0 mai 16, 2017 5:12:24 PM com.github.fommil.netlib.ARPACK WARNING: Failed to load implementation from: com.github.fommil.netlib.NativeSystemARPACK mai 16, 2017 5:12:24 PM com.github.fommil.netlib.ARPACK WARNING: Failed to load implementation from: com.github.fommil.netlib.NativeRefARPACK [16-05-2017 17:12:24] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 1 [16-05-2017 17:12:24] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 2 [16-05-2017 17:12:24] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 3 [16-05-2017 17:12:24] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 4 [16-05-2017 17:12:25] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 5 [16-05-2017 17:12:25] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 6 [16-05-2017 17:12:25] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 7 [16-05-2017 17:12:25] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 8 [16-05-2017 17:12:25] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 9 [16-05-2017 17:12:25] [OK] [Total Model] Started building a model over the full dataset. [16-05-2017 17:12:25] [INFO] [Upload Run] Starting send run process... [16-05-2017 17:12:25] [INFO] [SciMark Benchmark] Doing JVM Benchmark. [16-05-2017 17:13:01] [INFO] [Upload Run] Run was uploaded with rid 2160355. Obtainable at https://www.openml.org/api_new/v1/run/2160355 [16-05-2017 17:13:01] [OK] [Total Model] Done building full dataset model. Postprocessing... Done

Thank you very much in advance,

Best regards, Miguel

janvanrijn commented 7 years ago

Evaluation engine is a bit behind.

I will try to make it prioritze on recent runs in a bit.

Sorry for the inconvenience

2017-05-16 18:37 GMT+02:00 mcachada notifications@github.com:

Hi,

I've been performing some tests using OpenmlWeka in both GUI and command line. However, I'm getting "0 Evaluation measures" for my experiments. For example: https://www.openml.org/r/2160355.

Can you please help?

Call information and console output for the run mentioned above:

C:\Windows\system32>java` -Xmx8192m org.openml.weka.experiment.TaskBasedExperiment -T 1 -C weka.classifiers.lazy.IBk -- -K 5 -W 0 -A "weka.core.neighboursearch.LinearNNSearch -A "weka.core.EuclideanDistance -R first-last"" [16-05-2017 17:12:21] [OK] [Cache] Obtained from cache: task/task_1.xml Initializing... CrossValidationResultProducer: setting additional measures for split evaluator Iterating... [16-05-2017 17:12:21] [OK] [Cache] Obtained from cache: datadescription/ datadescription_1.xml [16-05-2017 17:12:21] [OK] [Cache] Obtained from cache: dataset/dataset_1.ARFF [16-05-2017 17:12:21] [OK] [Cache] Obtained from cache: splits/splits_1.arff class org.openml.weka.experiment.OpenmlClassificationSplitEvaluator [16-05-2017 17:12:24] [OK] [Attribtes] Attributes available: [family, product-type, steel, carbon, hardness, temper_rolling, condition, formability, strength, non-ageing, surface-finish, surface-quality, enamelability, bc, bf, bt, bw%2Fme, bl, m, chrom, phos, cbond, marvi, exptl, ferro, corr, blue%2Fbright%2Fvarn%2Fclean, lustre, jurofm, s, p, shape, thick, width, len, oil, bore, packing, class] [16-05-2017 17:12:24] [OK] [Class] Class attribute: @Attribute https://github.com/attribute class {1,2,3,4,5,U} [16-05-2017 17:12:24] [INFO] [Splits] Obtaining folds for Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0 [16-05-2017 17:12:24] [INFO] [Splits] Folds obtained [16-05-2017 17:12:24] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 0 mai 16, 2017 5:12:24 PM com.github.fommil.netlib.ARPACK WARNING: Failed to load implementation from: com.github.fommil.netlib. NativeSystemARPACK mai 16, 2017 5:12:24 PM com.github.fommil.netlib.ARPACK WARNING: Failed to load implementation from: com.github.fommil.netlib. NativeRefARPACK [16-05-2017 17:12:24] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 1 [16-05-2017 17:12:24] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 2 [16-05-2017 17:12:24] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 3 [16-05-2017 17:12:24] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 4 [16-05-2017 17:12:25] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 5 [16-05-2017 17:12:25] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 6 [16-05-2017 17:12:25] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 7 [16-05-2017 17:12:25] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 8 [16-05-2017 17:12:25] [INFO] [Perform Run] Started on performing Task 1 (anneal) with weka.classifiers.lazy.IBk - Repeat 0, fold 9 [16-05-2017 17:12:25] [OK] [Total Model] Started building a model over the full dataset. [16-05-2017 17:12:25] [INFO] [Upload Run] Starting send run process... [16-05-2017 17:12:25] [INFO] [SciMark Benchmark] Doing JVM Benchmark. [16-05-2017 17:13:01] [INFO] [Upload Run] Run was uploaded with rid

  1. Obtainable at https://www.openml.org/api_new/v1/run/2160355 [16-05-2017 17:13:01] [OK] [Total Model] Done building full dataset model. Postprocessing... Done

Thank you very much in advance,

Best regards, Miguel

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/openml/openml-weka/issues/3, or mute the thread https://github.com/notifications/unsubscribe-auth/ACL7-oH6R27lrSleCU3TpVT8U3WdKNtuks5r6dDjgaJpZM4NcxLf .

mcachada commented 7 years ago

Hi Jan, thank you.

How long would you say is the typical delay? Hours or days?

I'm about to run a bunch of experiments (+-1500) and need to know If I can rely on the evaluation measures to be available in "useful" time :)

Best regards, Miguel

janvanrijn commented 7 years ago

All supervised classification? I started a new thread of the eval engine focussing on this task. Should catch up soon. Let me know if tomorrow we are still behind.

2017-05-16 21:00 GMT+02:00 mcachada notifications@github.com:

Hi Jan, thank you.

How long would you say is the typical delay? Hours or days?

I'm about to run a bunch of experiments (+-1500) and need to know If I can rely on the evaluation measures to be available in "useful" time :)

Best regards, Miguel

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/openml/openml-weka/issues/3#issuecomment-301881552, or mute the thread https://github.com/notifications/unsubscribe-auth/ACL7-vbVKaKaqwfOIhZiUjiwktg1qKryks5r6fJDgaJpZM4NcxLf .

mcachada commented 7 years ago

Yes, all supervised classification.

FYI, my runs from 9 hours ago have already the performance measures available, but the ones from 5 hours ago don't, so the delay is in this 5-9 hours interval. I can work with that.

I'll let you know if the server is still behind tomorrow. Thanks again.

Best regards, Miguel