Other comments reviewer 1

[fbkarsdorp]

Other comments:

• [x] p. 1: "Increasingly, people interact with a variety of artificial agents, often even without being fully aware of whether or not their conversation partners are in fact human." ==> are you sure people are often not aware of this? This seems like an overstatement.

• [x] p. 4: Why did you decide to allow the same amount of time for A- and B-runs? Why not allow self-paced reading in both conditions?

• [x] p. 5: "encompassing the main body of English Hip-Hip music produced and consumed in the United States of America. " ==> ..English-language Hip-Hop.. Also: I think the "consumed in the United States of America" part could be removed, as this is now a global genre.

• [x] p. 5: since PLOS is a general scientific journal, it would be good to briefly spell out what LSTMs and Transformers are (with references and/or pointers to later sections of the paper)

• [x] p. 5: "we translated all unique words into a" ==> this slightly confused me; do you mean all words that occur only once (the hapax legomena), or all words that occur (all types)? Initially I assumed the former, but surely it must be the latter.

• [x] p. 6: "LSTMs have been shown to excel at Language Modeling [31] and we therefore resort to it" ==> .. resort to them

• [x] p. 6: "(i.e. in the present corpus from 89337 syllables to 172 characters)" ==> 172 characters is more than one might expect, so perhaps briefly explain where this number comes from

• [x] p. 6: "The reasoning is twofold: (i) noisy data.." ==> here, also, it might be worthwhile to say something about the possibly noisy input

• [x] p. 7: "extracting single word-level distributional feature vector." ==> .. vectors

• [x] p. 7: "One possibility to accomplish it is to initialize" ==> .. accomplish this..

• [x] p. 7: "Our model, however less general since it assumes.." ==> Our model, however, is less general (..) yet still achieves...

• [x] p. 8: "we fine-tune the on a model-per-model basis" ==> something wrong here

• [x] p. 8: "by manually inspection of the model output at different temperature values" ==> ..manual inspection.. More importantly: can you say a bit more about how this manual inspection was done?

• [x] p. 8: "Following the template, we generate as many sentences.." ==> it might be worth pointing out that templates are also often used in NLG (although arguably in a somewhat different way). See e.g., Deemter, K. van et al. (2005). Real versus template-based natural language generation: A false opposition?. Computational Linguistics, 31(1), 15-24.

• [x] p. 8: "where $\mu$ was selected per model through an inspection of random samples" ==> please briefly say how this was done

• [x] p. 9, caption table 5: "PC words have been deliberately masked" ==> I assume this should be Non-PC words, right? And what about motherfcking and sht? Also: what is a W model?

• [x] p. 11: "participants performed significantly words on" ==> ..significantly worse..

• [x] p. 11: "As can be observed from the marginal effects plot in Fig 2a, the learning effect is present in both question types and it is most strongly pronounced at the beginning of the game, after which it diminishes." ==> "most strong pronounced" is a pleonasm ("most strong" or "most pronounced"). More importantly: could this suggest that people start to pick up cues of neurally generated text (see above)?

• [x] p. 12: I would suggest removing \mu = and \sigma = and just report means and SDs, as 0.045 (0.057) (i.e., M(SD)), which is much more common.

• [x] p. 16: "At the same time, Hip-Hop lyrics very often do not develop longer stretches of thematically coherent narrative, ..." ==> I beg to differ. Do you have any evidence for this claim? If not, it would be good to phrase this a bit more cautiously.

• [x] p. 16: "This effect might also be reduced when longer fragments are admitted." ==> I agree, and think this would be a very interesting question for follow-up research. Maybe make this explicit?

• [x] p. 17ff: the references are not fully consistent in how they cite pages and dates. The Turing reference stands out because it is all caps. Would be good to make this consistent.

[fbkarsdorp]

There are some leftovers here for you, @emanjavacas

[emanjavacas]

I have touched the remaining points. Most of them were relatively trivial but for the LSTM I've added the equations. Could have some feedback on that.

[fbkarsdorp]

Great. I'll have a look tomorrow.