Add computation of z-maps

[aucuparia]

20 has a proof-of-concept.

To-do:

• Add an index (like filled.contour()).
• Maybe find a more appealing way to visualize the Z-map then the current bare-bones direct manipulation of the RGB image array. There are some packages and functions to plot (filled) contours, but so far these don't seem to work with raster images.
• Find a good way to do normalization. Currently we just normalize to -1:1 range. This makes images with different Z-maps impossible to compare because the same colour may represent different z-scores for different Z-maps. Quick solution: divide by 10 every time (assuming we won't have z-scores that high).

[rdotsch]

More to-dos:

• [x] Add progress bar
• [x] generateCI should by default not compute Z maps (due to the time it takes)
• [ ] add formal cluster tests on the z maps (see Chauvin2005)?
• [ ] output cluster test statistics (locations and size of the clusters)?

[aucuparia]

Re: default off. I think it's better to default to the 'quick' method (vectorize ci -> blur -> calculate z-scores using scale()). That one's very efficient.

[rdotsch]

It’s fast, but it doesn’t yield correct results. We’ll discuss it face to face.

On 26 Oct 2016, at 18:07, aucuparia notifications@github.com wrote:

Re: default off. I think it's better to default to the 'quick' method (vectorize ci -> blur -> calculate z-scores using scale()). That one's very efficient.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/HSWebteam/rcicr/issues/5#issuecomment-256397512, or mute the thread https://github.com/notifications/unsubscribe-auth/ADqoTGhdn_8kgqFml6r5grlrGhALUWF8ks5q33rMgaJpZM4KDpJb.

[aucuparia]

I know, but perhaps it's useful for a first impression. Or perhaps that'll just confuse people. We'll discuss.

[rdotsch]

It will be confusing and I want to protect users who don't RTFM. There is another consideration, and that is that we want the default behaviour to be identical to previous versions for backwards compatibility.

More necessary to-do's:

• [x] the z-maps computations are now correct for single subjects, but not for group CIs, where the t-tests should be performed on the set of computed CIs by subject, otherwise the error term is underestimated and as a consequence we get inflated z-values (i.e., more false positive pixels). Suggestion 1: create a generateGroupCI() function that wraps around generateCI() to solve this issue, or Suggestion 2: add a groupby argument to the function that takes a vector (e.g., with subject numbers) equal of the length to the stimuli/response vectors, which can be used to create images per subject and then perform t-test on those instead of all the individual stimuli.
• [ ] batchGenerateCI() should be able to take and pass the new arguments for generateCI() as well
• [ ] one of the most important use cases for z maps is actually for comparing two (group) CIs, to test where the significant differences are. In order to do this we need to do a two-samples t-test on the selected patterns for the two subject-level CIs (in the case of comparing two subjects) or a two-samples t-test on the subject-level CIs underlying the two group-CIs (in the case of comparing two groups). For 4AFC tasks, we need even a different implementation if we compare two CIs for different response options for the same participants on the group level, because there we'd need to use paired-sample t-tests.
• [x] customisable color gradient for the z maps
• [x] give user the choice to display (significant) z values on CI, on base image only, or pure z-map without underlying image.

We'll discuss all of this face to face.

[aucuparia]

raster::plot accepts custom colour palettes with the col argument (see ?raster::plot). We could explicitly add that argument to generateZmap, but I think it's sufficient to just mention it in Details or as an example in the description of the ... argument. I've mentioned it in Details for now.

[rdotsch]

• [x] Make zmap file name in generateCI() customisable, as we do with ci file name (perhaps reuse the already existing filename parameter)

[rdotsch]

Great work on generateZmap() so far! I've played around with it a bit.

• [x] I think smoothing is not yet implemented in generateZmap(). Is that correct? Yes
• [x] Perhaps a better name would be visualizeZmap(), because the actual z-map is generated in generateCI(). Changed to plotZmap
• [ ] We could also choose to take out the saveaspng part for the CI and create a separate visualizeCI() function, that gives the user more flexibility in visualisation options, including perhaps decorations.
• [x] The default color gradient for the raster::plot is very atypical when comparing it to neuroscience like visualisations. Proposal to use col=rainbow(100,alpha=.5, start=0, end=2/6) as default? Default set to viridis(100)
• [ ] I am going to read up on Chauvin et al. (2005) a bit more. The current method works great for finding significant pixels. But as Chauvin describes, sometimes we're looking for broad changes that don't yield individual pixels with z > 3, but still the pattern of a number of neighbouring pixels in unlikely due to chance. These are the cluster tests, and they are a follow-up analysis that work on the z-maps. Basically these look for unlikely combinations of slightly higher |z| values.

[rdotsch]

I am going to open up a separate issue for the cluster tests. Another todo for the z-maps visualization:

• [ ] Mask zmap (and optionally the bgimage)

[aucuparia]

Re smoothing: No, smoothing is in the Z-map part of generateCI. I think we should move the smoothing to visualizeZmap instead, but this can only be done when the smoothing is applied after calculating the Z-map. I'm fairly sure that's the way to go though, so we'll move it there eventually.

Re visualizeZmap: Agree, with the possible alternative of plotZmap. That's short, but I'm not sure if English allows us to call Z-maps plots.

Re visualizeCI: Could be useful, yes. Let's open another issue for that.

Re default colors: Your suggestion looks like a sane default to me, but let's consider using viridis. It's the coolest colour palette on the block. See https://cran.r-project.org/web/packages/viridis/vignettes/intro-to-viridis.html and this talk: A Better Default Colormap for Matplotlib

[rdotsch]

Re smoothing: I think there are two levels of smoothing. The Random Field Theory approach assumes smoothing to happen before stats like z and clusters are computed. We have to think that through at a later stage, once I get RFT working.

Re: plotZmap is better, indeed.

Re default colors: fine by me!

[aucuparia]

Filename can now be specified in plotZmap using the filename parameter. generateCI passes the value of baseimage to as the filename to plotZmap.

Proper importing and checking of the mask has been implemented as well. What remains to be done is the application of the Z-map.

A question: importing and applying of the mask is currently part of plotZMap. This is not a problem if we calculate the Z-map using our own method, but if we use quick (for example), the Z-scores per pixel are not independent any more. We either need to ensure that we don't implement Z-map methods in which Z-scores are dependent on other/all pixels, or we need to move the masking to generateCI.

[aucuparia]

Another question: we could re-use the mask importing code by putting it in a function. That way we can call importMask in generateCI as well as in plotZmap. On the other hand, this could be a case of unnecessary/unwanted code fragmentation and complexity. Your thoughts?

[rdotsch]

Re: function to import mask -- Yes, I am all for removing redundancy and put it in a function. I think it will reduce complexity.

Re: when to implement masking -- I am not sure I fully get your point.

[rdotsch]

generateCI now fails for me where it worked before this change.

Command:

s1.c0 <- generateCI(stimuli = study1.c0$ID, responses = study1.c0$response, participants = study1.c0$subject, baseimage = 'mnes', rdata = 'study1.Rdata', zmap = TRUE, zmapmethod = 't.test', saveaspng = TRUE, filename = 'group_ci_condition_0.png')

Result:

starting worker pid=5466 on localhost:11826 at 16:19:59.008 starting worker pid=5474 on localhost:11826 at 16:19:59.293 starting worker pid=5482 on localhost:11826 at 16:19:59.576 starting worker pid=5490 on localhost:11826 at 16:19:59.852 Loading required package: rcicr Loading required package: rcicr Loading required package: rcicr loaded rcicr and set parent environment Loading required package: rcicr loaded rcicr and set parent environment loaded rcicr and set parent environment loaded rcicr and set parent environment |==============================================================| 100% Error in plotZmap(zmap = zmap, bgimage = combined, filename = baseimage, : argument "mask" is missing, with no default

The problem seems to be that I don't/can't specify a mask as argument for generateCI to pass onto plotZmap. But here I want default behaviour without a mask at all.

[rdotsch]

The solution is to just not call plotZmap() from generateCI()at the very end. The user should do this themselves after calling generateCI() right?

[rdotsch]

I commented out the plotZmap() call from generateCI() for the time being in this latest commit. We can discuss how we want it to work in this thread and them always reinstate that line if we want to.

[aucuparia]

The mask argument of plotZmap indeed had no default value. This shouldn't have been a problem, because generateCI doesn't generate the Z-map by default, but there was an error there as well. Both are fixed now, so I've uncommented the call to plotZmap.

[rdotsch]

We should implement the approach to z-mapping, using the new masking functionality: take pixels that are masked away assuming these have no signal, then extract the distribution of those pixels from all selected stimuli across trials. Then z-score the pixels inside the mask based on this distribution.

As described in: Chauvin, A., Worsley, K. J., Schyns, P. G., Arguin, M., & Gosselin, F. (2005). Accurate statistical tests for smooth classification images. Journal of Vision, 5(9), 659–667. https://doi.org/10.1167/5.9.1

[rdotsch]

"However, if the number of trials is greater than 200 as is usually the case with classification images the transformation can be approximated as follows: where the mean and standard deviation are estimated directly from the data, preferably from signal-less regions of the classification images (e.g., regions corresponding to a homogeneous background)." from Chauvin et al. (2005).

[rdotsch]

• [ ] Need to check whether imported mask is processed accurately (specifically, coding of black/white ones and zeros).