added ebbinghaus_illusion, removed submodule and any .git

[pjkohler]

Can you reduce your jspsych folder to include only the files and plugins you use for the experiment?

[pjkohler]

Can we get rid of some of these folders that are not part of the specification? nda, ref, etc.?

[joshkenney]

Sure I can remove these folders. I would argue that the NDA definitions file is now required for all NIH-funded studies and worth including, but will remove for now.

What is the advantage to paring down the jspsych library?

[pjkohler]

The advantage of paring down is just to avoid bloating the repo with multiple repetitions of the jspsych.

What is the purpose of the NDA definitions file? I am unfamiliar. In any event I think that similar to consent forms, this would be highly dependent on the rules and regulations at each individual users location.

In general, my hope is to reduce these experiments to only include whatever is necessary to actually run the experiment and collect data.

[pjkohler]

Question: Is it possible to run this experiment locally? Which file would I use?

[joshkenney]

If trying to save on bloat, why don't we have all experiments reference a common jspsych folder at root?

All NIH-funded studies now require use of standard variables names that are required for submitting to the NDA repo. My experiments builds in these variables through the intake form. The nda/ directory stores the associated documents.

Right - but if people want to keep their experiments up-to-date, they should import jspsych as a submodule.

[joshkenney]

You can run index.php inside of htdocs via localhost using XAMPP

This experiment is designed to be hosted and just needs to be dropped into the web server root to run.

[pjkohler]

we deliberately decided to not use a common folder, because it would mean that all experiments would have to use the same version of jspsych, and in turn mean that repo maintainers would have to update experiments to comply with any future updates.

[pjkohler]

@joshkenney, it is great that your experiment is designed to be hosted, but I think that is somewhat beyond the scope of what we were considering here. I don't personally know a whole lot about hosting experiments myself, I have been using Pavlovia for that. I may switch to cognition.run in the near future, but I am not planning on self-hosting.

My imagined primary audience for this repo were people that were similar to or slightly to substantially below my skill level, like the psychology undergraduate and graduate students that tend to come to my lab.

@jodeleeuw, any inputs?

[joshkenney]

I've hosted the fork on my server so you can check out the experiment: https://stdftd.com/experiment-demos/ebbinghaus_illusion/

I just don't think hosting is all that hard and we should give people the tools to host for themselves, if the wish. It's all built into the framework of my experiments. They are also compatible with mTurk, prolific, etc.

A little background on me - BS in Psychology and completing MS in Computer Science. I'm happy to share what I know about task hosting. It's not the barrier that people think it is.

[joshkenney]

jsPsych is web development...I don't think we can really get around that fact.

[pjkohler]

jsPsych is web development...I don't think we can really get around that fact.

Not sure I agree, and even if it is, it does not mean that hosting is required to use jsPsych. Plenty of people use jsPsych without having much idea about hosting, I am one of them.

We should aim to reduce barriers for inclusion in the jsPsych community. I think services like Pavlovia and Cognition.run that take care of the hosting aspect is one way of doing that.

[joshkenney]

I disagree - I think this an opportunity to introduce web development to the psychology community. I've taught web development to multiple members of my lab and collaborating labs, many of them novices, and they have all been able to run with it.

As psychology becomes more computationally-driven, I don't see how sandboxing developers is a viable long-term strategy.

[pjkohler]

I think the community would appreciate a "how to get starting on self-hosting jsPsych" guide written by you. I certainly would. But that is beyond the scope of this particular repo.

[joshkenney]

Right on, I'm happy to do that. The guide would reference the framework I am suggesting be added to this repository.

If not in the scope of your current project, I could develop my own repo. However, I would like to adopt the standardization principles of this library.

Perhaps we could find some time to chat over zoom? It would help me to better understand where our goals overlap and how I can be most helpful.

[pjkohler]

I think a step-by-step guide for dummies on how to take a jsPsych experiment that can be run locally and uploaded to cognition.run or Pavlovia.org, and "convert" it to self-hosting would be really great. Not sure you would need a repo for that, but maybe ...? I think adopting the standardization practices (as we invent them) for this library would be great, and I also think you should consider simplifying any example code you use for your guide as much as possible. You have a decent amount of complexity in your code, which may confuse your target audience (dummies like me).

[pjkohler]

I sent you an email about a meeting.

[jodeleeuw]

I think one key idea behind jspsych is transferability across backend frameworks. So one person can self host and another can use cognition.run. The framework is deliberately agnostic about this.

(accidentally hit close on my phone keyboard while I was typing this, oops!)

Given this principle of transferability, I think it would be ideal if tasks in this repo are also agnostic about backend implementation. Having guides about how to host these tasks in conjunction with code that is hosting agnostic would be even better.

[joshkenney]

@joshkenney yes, my wrapper is similarly agnostic. The goal is for all experiments to run happily wherever they live.

[jodeleeuw]

But if it includes server side php it's not fully agnostic, right?

[joshkenney]

Right, but you could just run AJAX instead to render it client-side. I'll work on that implementation.

I put if off because I can't imagine why anyone would not want to have their data save to a .csv file.

What do people do instead? Copy and paste the text output after each experiment?

[jodeleeuw]

There are lots of alternatives to a particular PHP -> .csv file pipeline. I sometimes use a MySQL database and sometimes I dump .json files directly to the server disk. I also sometimes use cognition.run, where I don't need to set up the backend at all and data files are stored for me automatically. Lots of folks like Pavlovia, JATOS, or Gorilla, which all have their own APIs for saving to disk. And sometimes I run jsPsych experiments in the lab and just use the .localSave() function to download json at the end of the experiment.

[joshkenney]

Thanks for sharing your experience with these different technologies. Of course, everyone will do things differently. The ideal wrapper would allow for these different saving methods.

For the purposes of this repo, how would you like the data to be saved?