Matlab logging

[mangstad]

as discussed with @dankessler we need to write a utility function that will create a timestamped copy of the template script that was run so that data can easily be audited if necessary. This can potentially also include the sha id of the current version of the methods core repository being used so that exact results can be recreated if required.

Additionally, this function will work with an error handling function to log any matlab errors that occur.

[dankessler]

Assigned to @mangstad

[rcwelsh]

I was thinking about this as I was walking across campus. I think we should also provide some sort of auditing ability of image data in addition to scripts.

Presently time-stamping is done already in spm8Batch to both the scripts as well as the data that is touch and I'll be adding quite quickly additionally centralized logging/time-stamping of all processing that touches data for a given subject. that is, presently the logging/timestamping available in spm8Batch writes out time-stamps for each process into a "hidden" file at the same location as the data it operated upon.

look at my matlab script : UMBatchLogProcess

which is called at the end of any image processing.

additionally, there is a bash logging routine as well: logTheUMProcess

ADDED NOTE ABOUT THE TIMESTAMP: In addition the the SHA id, there should also be a human-readable/human-understandable version comment for investigators to make sense.

[dankessler]

For public releases, it makes sense to have an incremented "tag" number, which can be the human-readable version info printed out. Since public releases are likely to be less frequent, it won't be too hard to keep track of this.

However, if somebody wants to work from a development branch or some sort of incremental release, then the SHA id is probably as much detail as they'll get.

[rcwelsh]

one thing, seems as though the solution we had working on my machine to pick up the SHA from that "hidden" file ".spm8BatchVersion" might have broken when moving to the central repository. we can look at it next week. maybe it's something i'm doing stupidly?

[dankessler]

No, you're right. This support doesn't yet exist in the central repository.

I'm adjusting the hooks to work for everything in #29, not just spm8Batch.

Instead, the current SHA ID will be in a file called .local/CurrentVersionSHA, but this fix isn't in the develop branch yet.

On Fri, Mar 9, 2012 at 4:14 PM, Robert < reply@reply.github.com

wrote:

one thing, seems as though the solution we had working on my machine to pick up the SHA from that "hidden" file ".spm8BatchVersion" might have broken when moving to the central repository. we can look at it next week. maybe it's something i'm doing stupidly?

---

Reply to this email directly or view it on GitHub:

https://github.com/UMPsychMethodsCore/MethodsCore/issues/27#issuecomment-4424795

Daniel A. Kessler Research Area Computer Specialist Psychiatry - Rachel Upjohn Building University of Michigan, Ann Arbor kesslerd@umich.edu +1 734.418.8134

[dankessler]

@rcwelsh Sorry for breaking this functionality when I ported spm8Batch to the unified repository. There are some neat features with the hooks though that are coming soon, check out #29 :)

[rcwelsh]

@dankessler no worries. i'll touch base on this in person on friday. i'm finalizing the testing of my spm8Batch now for the next release. working on changing some of the error handling. though it would be nice to incorporate any from #28

[dankessler]

Snagged from an email from @mangstad to @sripada @heffjos

Yeah, for the logging, my intention is to add archiving and logging to all the central scripts (though can be limited to whatever is >being released right now for now). So each time you run a template script, it will save a timestamped copy of itself in a log >folder. Then while that processing is running, any calls to mc_Error will also write the output to an error logfile with the same >timestamp in the log folder (something like Preproccess_template_20120605_125432994.m and >Preprocess_template_20120605_125432994.err or .log or something).

The idea is to do something like this:

[st i] = dbstack('-completenames');
callingscript = st(end).file;
[p f e ans] = fileparts(callingscript);
rightnow = now;
scriptcopy = fullfile(logdirectory,[f '_' datestr(rightnow,'dd-mmm-yyyy_HHMMSS') e]);
scripterr = fullfile(logdirectory,[f '_' datestr(rightnow,'dd-mmm-yyyy_HHMMSS') '.err']);
shellcommand = sprintf('cp %s %s',callingscript,scriptcopy);
[status result] = system(shellcommand);
if (status ~= 0)
      mc_Error(result);
end

This grabs the name of the calling script (should be the template script) then creates a copy of the script in a log directory (have to add something like a LogTemplate to the template scripts) and then generates the name of an error log file (scripterr). The idea that Daniel and I had was to use a global variable like mc_Log or something to store the name of the error log script, then the mc_Error function when called would write the error to this file.

[dankessler]

Closing this now that pull request #66 has been adopted