svmbatch needs a matlab frontend

[dankessler]

There needs to be a template/central script structure built for svmbatch.

Some existing features need to be ported

• specify paths to templates
• Perform regular LOOCV (leave one out cross-validation)
• Perform "super" LOOCV (LOOCV by hand where decision function value for each LOOCV is stored)
• Totem stacking to combine images

Once built, there are some additional features that can be more automated

• Automatic write-out of model performance (otherwise sent to the terminal)
• Extraction of decision functions from cross-validation, and write-out to csv with meaningful subject names
• Permutation testing based on already written-out permutation models and weight vectors
• Masking support

  Other stuff to add? @sripada and I just talked about this but I'm not sure what I've forgotten.

[rcwelsh]

@dankessler ( @sripada )

Can I ask the obvious question, why the extra layers? If you are going to write a wrapper to the already wrapped AFNI calls why not just do it directly? I'm concerned that we are adding too many layers upon layers, or that is, using up our time to write wrappers of wrappers?

[dankessler]

@rcwelsh A fair question. At present, the svmbatch toolbox that I wrote is entirely in bash. It makes sense to me, but would not be easy for another user to run. I'm not planning to write a wrapper to my existing bash scripts, but to instead port them to use matlab wherever possible (and to call bash from matlab wherever not possible). The idea is to make running svmbatch as straightforward as running firstlevels using the "MikeFirstLevel" approach, so that any of the investigators who already do first levels could, just as easily, add SVM to their functional repertoire. Of course, understanding what goes into and comes out of SVM is another matter, but I want to make the code as easy and standard as possible.

[rcwelsh]

@dankessler What I do NOT like about this solution is that it consumes a MATLAB license for no purpose. Given that all of AFNI is C/C++ based then any wrappers should stay in BASH. Better to have some sort of "config" text file for running an analysis. Certainly, post analysis statistics can be done in matlab/spss/R, but truly if the calculation work is being done in matlab. Using up matlab licenses just as a layered scripting language is not the best use of resources. In the long run it's better to have a unified API that can be called from the command line. people use the matlab command line, they can learn the bash command line, and use gedit instead of the matlab editor to edit text files -- another non-essential use of a matlab license.

[dankessler]

@rcwelsh I understand the objection and definitely don't want to encourage the unnecessary consumption of MATLAB licenses. Maybe this is a policy issue we should discuss as a group at more length on Friday. I'm not planning to invest any time in working on this issue before then, anyway.

[rcwelsh]

@dankessler Sounds good. I'd rather have you using your time creating/honing new methods rather than gift-wrapping already established methods.

[sripada]

@dankessler @rcwelsh

I definitely see your line of thinking Robert. But there are a couple of points that go the other way.

1. We are obligated to matlab front ends on some things, like our existing FirstLevel and SecondLevel scripts, which are used by most panlab PIs. Plus a host of related scripts for smaller tasks that Mike and I have made. They all share a common frontend, so I think its reasonable to standardize around that.
2. The Matlab license issue has not been an issue since Dan and I and our crew started using a single SLAB username. I think we can finesse user licenses with work arounds like this.
3. The gift wrapped front ends are not that time consuming to make. And they save a LOT of Dan and Mike (and Joe's) time b/c people can run and debug their own stuff, and training is kept to an absolute minimum (a Methods COre mission is dissemination after all). I think if we get frondends right and add verbose error messaging in the manner you've (Robert) suggested, then we will have a lot more time to focus on development without dealing with mickeymouse issues all the time.

But you are right, let's not discuss this here and instead save it for Friday. :)

[rcwelsh]

Not sure if you saw this:

http://afni.nimh.nih.gov/afni/download/afnimatlab/psc_project_view

[dankessler]

Yeah I've played with it a bit. It's pretty handy :)

However, it doesn't look like they've implemented the SVM stuff there.

[dankessler]

I'm leaning towards pulling svmbatch out of the Methods Core repository and developing it independently. Right now I'm not sure how much demand there is for svmbatch capabilities, and I'm not sure we want to support it as a core for now.

If I do that, this issue is becomes irrelevant.

[rcwelsh]

mmmm, you realize you are adding to my argument of having projects as separate repositories :-D

[dankessler]

svmbatch is now getting some significant revisions to directly call the svm_light library from matlab which is enabling much faster and generalized svm applications. The eventual release will include a matlab front-end for these applications which bypasses 3dSVM and directly calls svm_light.

[rcwelsh]

@dankessler hey, thanks for finding this, i just saw it to, might do the same thing as it'll save some disk space from having to make briks. though does it handle all of the different options of 3dsvm? (i'll look of course myself.)

[rcwelsh]

more thoughts, lets talk about this when I'm back. i have some functionality that i'd like to see in place:

1) using multiple time-series data -- from different locations (and NIFTI files) 2) being able to specify a censor vector (3dsvm has that) 3) indicating where the model should be written

things like that. the major hangup I've got about 3dsvm is that all data etc has to reside in the same directory etc. my preference if it's possible assuming that the matlab interface is used is more and more flexibility.

are you using the mex frontend? that would be better maybe so that temporary files can be avoided?

[dankessler]

Closing this issue as svmbatch now uses matlab calls for model learning and other things