Review Request: Le Masson, Alexandre

[maekclena]

AUTHOR

Dear @ReScience/editors,

I request a review for the replication of the following paper:

• How Attention Can Create Synaptic Tags for the Learning of Working Memories in Sequential Tasks, J. Rombouts, M. Bohte and P. Roelfsema, PLoS Computational Biology, 11, 2015.

I believe the original results have been faithfully replicated as explained in the accompanying article.

Repository lives at https://github.com/maekclena/ReScience-submission/tree/Le_Masson-Alexandre-2016

---

EDITOR

• [x] Editor acknowledgment (@oliviaguest 09 August 2016)
• [x] Reviewer 1 (@vitay 12 August 2016)
• [x] Reviewer 2 (@eroesch 8 September 2016)
• [x] Review 1 decision [accept] 30 November, 2016
• [x] Review 2 decision [accept] 1 December, 2016
• [x] Editor decision [accept] 1 December, 2016

[oliviaguest]

Thanks @maekclena I will assign reviewers soon.

[vitay]

The authors reproduce successfully the results of the original paper on 2 of the 4 proposed tasks. The other tasks (category matching and vibrotactile discrimination) would not bring much to prove that the model is correctly reproduced, so this is fine. The model itself is actually quite simple, reproducing the tasks (especially the probabilistic task) seems to have been the main difficulty in the reimplementation.

There were no dependency problem to run the simulations, everything is standard and runs well (tested under linux). The code for the model is very well organized, clear and commented. There are a few docstrings missing in the classes, but the names of the methods are self-explaining anyway.

article

The article is well written and describes the difficulty of the reimplementation. A few words on the implementation itself could have been nice: why classes for the populations and projections instead of plain numpy arrays (what could have been slightly more readable)? Does reduce bring much in terms of speed, or is it just more generic (there are never more than two incoming projections to a population in the current model, but it could be beneficial for extensions)? Things like that.

In the results section, you do not provide the setup for computing the success rates and convergence speed. How many networks were used? What is the variance?

You could also describe slightly more in the text in how far Figs. 1 and 2. reproduce the original figures: it is not obvious without reading the original article that fixate dominates until the go signal, after which the correct action is chosen. Don't feel like you have to do it, but it would have been also nice to reproduce other figures, such as 2C or the upper part of 2D and 4C.

It might also be useful to clarify the nature of the feedback weights w' in equations 14 and 16: once an action is selected, only the feedback synapses leaving the corresponding selected Q-value unit are activated to update tags, more precisely: w' ij = w ij × z i . The model could also have dedicated feedback connections but the simpler method is to use the feedforward synapses' weights.

It is indeed not clear in the original article whether the feedback weights are also learned (as they seem to imply), or if they just copy the feedforward ones. What was used in the original implementation? So much for the biological plausibility: as the weights are randomly initialized, there is no chance that the feedforward and feedback weights take the same value in this learning setup. So the superiority over backpropagation is not that obvious...

Small typos:

• Introduction: it helped to verify the correctness
• Tasks: extra step to give the final reward
• Tasks: A number of settings is randomized

code

• In README.md, the usage should be described as:

python3 simulation.py -h...

as the scripts are not directly executable.

• It could be useful to specify which version of Python, Numpy and Matplotlib are needed (at least by me Python 3.4.3, Numpy 1.8.2 and Matplotlib 1.3.1 work). As you use the new package statistics of python 3, 3.4 is likely the minimal version.
• It would not cost much to make the code compatible with python 2.7 (still the default on popular distributions). You just need to add:

from __future__ import print_function

at the beginning of simulation.py and plot-activation.py, and to use the numpy implementation of median instead of statistics in simulation.py:

# from statistics import median
from numpy import median

[oliviaguest]

Many thanks to @vitay for the review. I will have to locate another 2nd reviewer, so please bear with me @maekclena while I try to track somebody down.

[oliviaguest]

EDIT: Please see below.

Would either of @MehdiKhamassi or @benoit-girard be able to take on the mantle of 2nd reviewer? :smile:

[eroesch]

Hi @oliviaguest; I'd be interested to do the review.

[oliviaguest]

Excellent! Go ahead!

[benoit-girard]

You are too fast for me! Next time...

[oliviaguest]

Any updates on this @eroesch?

Also have you had any time to address some of the above @maekclena?

[maekclena]

No, I haven't had time yet. I will wait for @eroesch's review to make all necessary corrections at once.

[eroesch]

I am almost done.

[oliviaguest]

@eroesch 🔔

[rougier]

Don't pay attention to the conflict, I just updated the submission directory. No incidence at all on this submission. @oliviaguest @eroesch Any chance for an update ?

[rougier]

@eroesch Do you think you can complete your review by the end of the week ?

[eroesch]

The submission presents a novel implementation of a model describing a new rule for reinforcement learning [1]. The authors review aspects of the implementation that appeared important when reproducing the work, and describe results obtained in similar setups. They reproduce 2 of the 4 tasks used in the original paper. I have managed to run the simulation (macos, Anaconda), and I am satisfied with the code provided, which is actually much shorter than I had anticipated. I thank the authors for their time—and their patience! Find a few comments below, which I believe will help making the submission clearer.

• Please provide more information on the implementation of the original model [1]; i.e. Which programming language? What particular libraries have they used (and do not forget versions)? Are there specific aspects of the development that are worth mentioning, like GPUs, version of CUDA for ANNarchy, etc?
• Are there comparative measures of performance between [1] and the implementation that you are presenting? For instance, you mention in the readme that computation time can be long. How does your implementation fare against that of [1]? Are there memory constraints that need to be taken into account?
• Also, I think it would be important to review the main differences between the two implementations that are due to the choice of programming language, especially as you mention that there are performance differences; in particular, I am mindful that differences in numerical accuracy, truncation policies, etc, would yield significant differences in difference ranges of parameters.
• Can you add information on how to get hold of the original implementation for [1]? (if on a public repo somewhere, please provide the URL)
• Please add more information about the versions of the packages that you have used.
• I imagine that if it would be indeed nice to add some of the above information in the text (more or less concisely), it could also feature in a dedicated table.
• In line with my previous comments, I find the discussion about the discretisation of time events extremely interesting, for the purpose of reproduced results. I would encourage the authors to add more on that topic, and especially the consequences of such apparently innocuous behaviour for the repro of the original results.
• Mentioning the “ambiguity” of the memory traces is important, please describe whether this is also a behaviour found in the original [1] or something due to your new implementation.
• I agree that the original submission was a bit unclear when describing the tasks, and thank the authors for providing more information, which were helpful to me. I would encourage the authors to expand if necessary. Do not hesitate to be critical and to discuss limitations, in light of the re-appraisal work that you engaged in.
• Ideally, you would provide enough information about [1] for a modelling-literate reader who has not yet read [1] to understand the what, why and limitations of the original work, and that of your own work. I would encourage the authors to think of their submission as the first point of entry to [1], rather than a side lined piece of evidence which someone might choose to look at after having gone through [1].
• In the Results section, please provide information on the number of runs, etc. Incongruence between the results is to be expected because of the stochastic nature of some of the computation, and it would help the reader to know what the percentages you provide actually mean.
• State clearly that you have followed strictly the original parameters (by referring to Table 1, 2, and 3 and adding any information that you may have found in the documentation of their code, not present in the article). E.g. in add a table with the parameters used.
• I would personally find it very interesting to compare the two implementations in different parameter ranges, to sort of home down to the gist of the differences in results, but that may be beyond the scope of the article.

[1] J. Rombouts, S. Bohte, and P. Roelfsema. “How Attention Can Create Synaptic Tags for the Learning of Working Memories in Sequential Tasks”. In: PLoS Computational Biology 11.3 (2015), e1004060. doi: 10.1371/journal.pcbi.1004060.

[oliviaguest]

@maekclena you probably want to read above reviews if you haven't already.

[oliviaguest]

Hey @rougier do you know @maekclena's email address? Perhaps we need it to get in contact?

[rougier]

On the article (PDF), there is the email of the corresponding author.

[falex33]

our response is being prepared... Very sorry for being late and thank you for your patience...

[maekclena]

Sorry for the delay and the lack of communication. We made the necessary corrections with respect to the reviews.

Most notable changes:

• scripts now support Python 2.7
• added requirements' versions
• added a link to the author's implementation
• added an implementations comparison section
• added more comments in the code

As well as additional details and rewritten sentences in the article.

[oliviaguest]

Reviewers — @vitay & @eroesch — are you satisfied with the changes that have been made? If yes, can you please formally accept the paper?

[vitay]

I am for accepting the paper, the changes answered my questions.

[eroesch]

I am accepting the paper, the changes answered (most of) my questions.

[oliviaguest]

Dear @falex33 and @maekclena — Congratulations! How Attention Can Create Synaptic Tags for the Learning of Working Memories in Sequential Tasks has been accepted! Publication and more details will soon follow.

[oliviaguest]

Hi all — I have attempted my first go at this:

[pdebuyl]

Hi @oliviaguest

The archived PDF has wrong links for the red buttons "Article repository", etc. on the first page.

The first one, for instance, points to https://github.com/ReScience-Archives/Le_Masson-Alexandre-2016/article whereas the "good" link would be https://github.com/ReScience-Archives/Le_Masson-Alexandre-2016/tree/master/article

[oliviaguest]

Sorry about this. No idea how to fix this problem unfortunately as this is the first time I've done this. I won't try anything until somebody who knows more advises. Perhaps @rougier knows more?

[pdebuyl]

The links are in the header of the .md file for the paper. What I don't know is how to manage the update to Zenodo, if one is done at all.

Sorry @rougier , we'll need an opinion :-)

[rougier]

I think you need to regenerate the right PDF and upload a new version to Zenodo (while keeping the same DOI).

[oliviaguest]

Indeed. This is why I asked for your more experienced opinion. So unless I am mistaken, which is possible... I'm not sure that I can change any files without emailing Zenodo. I am of course more than happy to email them myself to fix this problem!

Note: File addition, removal or modification are not allowed after an upload has been published. This is because a Digital Object Identifier (DOI) is registered with DataCite for each upload. If you've made a mistake please contact us.

[rougier]

Can't you release a new version on Zenodo using the same DOI ? If not, we'll have to issue a new DOI and update the website and ReScience repository.

[oliviaguest]

I'm unfamiliar with Zenodo but I don't think so...?

[rougier]

Just checked and you'r right. We'll have to contact Zenodo. Can you handle it ?

[oliviaguest]

I think before emailing we/I should fix the repo which we can edit ourselves first. So I need to edit the md file to correct the link, and recompile the pdf. Is there anything else that needs updating? Does the release on github allow changes?

[rougier]

No, I think you will have to make a new release (and double check links before making the release). I can help you on checking the PDF.

We really need to automate this part to avoid future errors. I did the same for one submission but I was lucky enough to realizd something was wrong before submitting to Zenodo.

[oliviaguest]

OK, I'll try on Monday!

[khinsen]

Zenodo lets you upload a new file and declare that it renders an earlier one obsolete. That's probably the best and most transparent way to handle corrections. If I remember correctly, you have to

1. upload and publish the new version, and refer to the old one in the metadata as "replaces..."
2. edit the metadata of the old version to add a reference to the new one.

[pdebuyl]

I see

is new version of this upload

Is this a kind of "official" replacement method? Will there be two DOIs?

[oliviaguest]

@khinsen could you point me to where that option is, please?

[khinsen]

It's under "Related/alternate identifiers". There you can enter URLs or DOIs of related things and assign a role to them. The two relevant roles are "is new version of this upload" and "is previous version of this upload".

For an example of how this is presented to the reader, see this software upload. Under "related identifiers" in the right column you can see both "previous versions" and "new versions".

@pdebuyl: Each upload has its own DOI. There are just annotated links between the two uploads. It's more of a versioning than a replacement method, but that's how the scientific record should work in my opinion.

[oliviaguest]

OK, I am attempting step one now of @khinsen's solution. Can somebody check that I do the github-side of things correctly please before I do anything on Zenodo? Thanks for the help so far. Will post a link when I have compiled the corrected .md.

[oliviaguest]

OK, so I think this has now fixed the link problem: https://github.com/ReScience-Archives/Le_Masson-Alexandre-2016/tree/master/article Is the next step a new release on github?

[rougier]

I think a link for the notebook in the PDF while there is no notebook, do you know why ? Was it the case with previous version ?

[oliviaguest]

Better? :smile:

[pdebuyl]

As the article is in re-processing (okay, because of me :-/) is it useful to have the link to data? It has not been used by the authors and contains only the template's README and LICENSE.

[oliviaguest]

Removed it. Is the repo and article looking done now? Do we need the data and notebook directories?