Failing to run from Weka Explorer and commandline

[cibic89]

Hi,

I would like to run this in WEKA. I am interested in training a convolutional neural network classifying images. In CLI, the iterator convolutional looks for instances whereas the image dataset iterator (where the directory with folders as labels and images within) gives an error looking for -t training file (WEKA). How do I set this up properly? If the latter is the proper direction, the training file is a list of complete image file path and class?

In Explorer it says "cannot handle binary class" regardless of how I try to set it up.

[christopher-beckham]

Hi,

Could you copy and paste the output/error from the CLI here? To answer your question, yes, the image iterator needs an ARFF where each row denotes the string path to the image and its class.

Just a side note too: the DL4J package in the official WEKA repository is slightly upgraded from this one here on Github. I was too busy last year to continue development for this on Github, so Eibe took over and cleaned things up (but that code has not been merged in this repository). In the coming months I would like to merge his code with mine here on Github and continue some work on this.

[cibic89]

Hi,

That explains a lot. Here, https://deeplearning4j.org/simple-image-load-transform it was suggested sorting the training images in folders named after the class labels. One does not need that for this scenario. I can have one folder with all images in and then point to each image in the arff file. Right?

The CLI error is the standard one received when weka cannot find the training file:

Weka exception: No training file and no object input file given.

General options:

-h or -help Output help information. -synopsis or -info Output synopsis for classifier (use in conjunction with -h) -t Sets training file. ...

Thank you for your efforts Christopher!

George

---

From: Christopher Beckham notifications@github.com Sent: 25 May 2017 14:21 To: christopher-beckham/wekaDeeplearning4j Cc: cibic89; Author Subject: Re: [christopher-beckham/wekaDeeplearning4j] Failing to run from Weka Explorer and commandline (#11)

Hi,

Could you copy and paste the output/error from the CLI here? To answer your question, yes, the image iterator needs an ARFF where each row denotes the string path to the image and its class.

Just a side note too: the DL4J package in the official WEKA repository is slightly upgraded from this one here on Github. I was too busy last year to continue development for this on Github, so Eibe took over and cleaned things up. In the coming months I would like to merge his code with mine here on Github and continue some work on this.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/christopher-beckham/wekaDeeplearning4j/issues/11#issuecomment-304008559, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AVnF90JgV52cGM4jGlxnKS1g664DK5Yhks5r9YBrgaJpZM4Nlvpd.

[christopher-beckham]

Yes, you can have all the images in one folder.

I have an example here:

https://github.com/christopher-beckham/wekaDeeplearning4j/blob/master/datasets/mnist.meta.minimal.arff

Note this ARFF is referring to the images here:

https://github.com/christopher-beckham/wekaDeeplearning4j/tree/master/datasets/mnist-minimal

If you try your arff (or this arff instead), does it work? Or do you get the training file error?

[cibic89]

Hi Christopher,

Tried this config in WEKA explorer and soon as it starts it stops with no error message:

weka.classifiers.functions.Dl4jMlpClassifier -S 1 -iterator "weka.dl4j.iterators.ImageDataSetIterator -height 28 -imagesLocation C:\mnist-minimal -numChannels 1 -bs 1 -width 28" -layer "weka.dl4j.layers.OutputLayer -activation \"weka.dl4j.activations.ActivationSoftmax \" -adamMeanDecay 0.9 -adamVarDecay 0.999 -biasInit 1.0 -l1Bias 0.0 -l2Bias 0.0 -blr 0.01 -dist \"weka.dl4j.distribution.NormalDistribution -mean 0.001 -std 1.0\" -dropout 0.0 -epsilon 1.0E-6 -gradientNormalization None -gradNormThreshold 1.0 -L1 0.0 -L2 0.0 -name \"Output layer\" -lr 0.01 -lossFn \"weka.dl4j.lossfunctions.LossMCXENT \" -momentum 0.9 -rho 0.0 -rmsDecay 0.95 -updater NESTEROVS -weightInit XAVIER" -logFile C:\mnist-minimal\log.txt -numEpochs 10 -algorithm STOCHASTIC_GRADIENT_DESCENT -output-debug-info

The log file does not get created from D4Lj and WEKA's log just shows the command I executed (above).

George

---

From: Christopher Beckham notifications@github.com Sent: 26 May 2017 00:02 To: christopher-beckham/wekaDeeplearning4j Cc: cibic89; Author Subject: Re: [christopher-beckham/wekaDeeplearning4j] Failing to run from Weka Explorer and commandline (#11)

Yes, you can have all the images in one folder.

I have an example here:

https://github.com/christopher-beckham/wekaDeeplearning4j/blob/master/datasets/mnist.meta.minimal.arff

[https://avatars3.githubusercontent.com/u/2417792?v=3&s=400]https://github.com/christopher-beckham/wekaDeeplearning4j/blob/master/datasets/mnist.meta.minimal.arff

christopher-beckham/wekaDeeplearning4jhttps://github.com/christopher-beckham/wekaDeeplearning4j/blob/master/datasets/mnist.meta.minimal.arff github.com wekaDeeplearning4j - Weka DL4J. Original code by Mark Hall

Note this ARFF is referring to the images here:

https://github.com/christopher-beckham/wekaDeeplearning4j/tree/master/datasets/mnist-minimal

[https://avatars3.githubusercontent.com/u/2417792?v=3&s=400]https://github.com/christopher-beckham/wekaDeeplearning4j/tree/master/datasets/mnist-minimal

christopher-beckham/wekaDeeplearning4jhttps://github.com/christopher-beckham/wekaDeeplearning4j/tree/master/datasets/mnist-minimal github.com wekaDeeplearning4j - Weka DL4J. Original code by Mark Hall

If you try your arff (or this arff instead), does it work? Or do you get the training file error?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/christopher-beckham/wekaDeeplearning4j/issues/11#issuecomment-304147278, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AVnF9-1uOYfcckeRhEgsn2VrYk_0hl3Mks5r9ghzgaJpZM4Nlvpd.

[christopher-beckham]

Judging by that command it looks like you're using the DL4J in the WEKA package manager (which is an upgrade from what is here on Github), since there are some classes I don't recognise, like weka.dl4j.activations.ActivationSoftmax... Are you able to also post this issue on the WEKA mailing list here?:

http://weka.8497.n7.nabble.com/

[cibic89]

Sure. Thank you Christopher!

---

From: Christopher Beckham notifications@github.com Sent: 26 May 2017 21:54 To: christopher-beckham/wekaDeeplearning4j Cc: cibic89; Author Subject: Re: [christopher-beckham/wekaDeeplearning4j] Failing to run from Weka Explorer and commandline (#11)

Judging by that command it looks like you're using the DL4J in the WEKA package manager (which is an upgrade from what is here on Github), since there are some classes I don't recognise, like weka.dl4j.activations.ActivationSoftmax... Are you able to also post this issue on the WEKA mailing list here?:

http://weka.8497.n7.nabble.com/

WEKA | Mailing List Archivehttp://weka.8497.n7.nabble.com/ weka.8497.n7.nabble.com WEKA forum and mailing list archive. WEKA machine learning software discussion (Make sure that you are registered with the actual mailing list before posting.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/christopher-beckham/wekaDeeplearning4j/issues/11#issuecomment-304386007, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AVnF90K2qLJJ0ghQ2OMzlLpSWG8P_6m5ks5r9zv8gaJpZM4Nlvpd.

[braun-steven]

@cibic89 Could your issue be resolved?

[cibic89]

Hi,

Yes I did get it resolved in the end.

Sent from BlueMailhttp://www.bluemail.me/r?b=10777 On 11 Oct 2017, at 01:44, Steven Lang notifications@github.com<mailto:notifications@github.com> wrote:

@cibic89https://github.com/cibic89 Could your issue be resolved?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/Waikato/wekaDeeplearning4j/issues/11#issuecomment-335648399, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AVnF90e5Hg7ttrpj3FzRz6vjT5M69esEks5srA9-gaJpZM4Nlvpd.