Can't port to new expfactory framework - Githubissues

earcanal / attention_orienting

Posner-ish attention orienting task

MIT License

0 stars 0 forks source link

Can't port to new expfactory framework #1

Open earcanal opened 6 years ago

earcanal commented 6 years ago

Continuing the tradition of communicating via github coments @vsoch.

I think I could have implemented this using the new framework, but I bottled out because that framework currently generates JSON rather than CSV output. Because the task is being used in a real study and I'm short of time, I'd rather not tackle this conversion right now. However, I think I will be able to port this one once

a) the framework itself can write experiment data as CSV b) I find a pattern to do JSON->CSV

vsoch commented 6 years ago

What do you mean pattern? Is there any reason that you couldn't convert to csv as part of your analysis step?

vsoch commented 6 years ago

And I'd be happy to port this for you, after we figure out the data format stuffs :)

earcanal commented 6 years ago

None at all. That's what I meant in b). Is it cleaner to keep file endpoints in JSON on the app side?

No, I want to try and port it. You have better things to do :)

vsoch commented 6 years ago

haha ok :) Well JSON == "JavaScript object notation," so (and this is my opinion) it's a much more browser native format than say .csv. It also makes it more friendly to restful APIs and places no constraints on the data structure (or lack of it). Something ilke csv relies on having a structure like a relational database (e.g. tables with headers) and json is more akin to a non relational database (no structure). And since you can easily convert to csv from json, and there are likely many surveys that wouldn't fit nicely into a table, I have (thus far) been using json. I think it makes sense to maintain. After you get a data structure to look at, I can offer help to transform it. Another idea is that individual experiments could have "helper" scripts for the user after the fact, to help convert to different data types, or even do analysis. What do you think?

earcanal commented 6 years ago

Yeah, I get that JSON makes sense inside a REST pipeline, but I think the framework should include CSV as an endpoint data format. There will be a class of users who won't engage if they have to implement the JSON->CSV step on the analysis side. It would be nice to be able to interact with TEF in terms of components (experiments and surveys) and output formats (CSV, database) for the class of users who don't want to touch any code at all.

I so need to draw a picture of what I think this project is, and where it's headed (partly for my own benefit). I think I've drunk the container kool-aid, but I'd like to check that before my vision starts to blur.

vsoch commented 6 years ago

Agreed! I have database as a variable, and have intended to add other format types (as user's request). The issue is how to best implement - if I place the burden on the creator of the template, this means that a user that wants a different format has a hard time getting it. On the other hand, if I make it more flexible to always require submission of the form as json (and then do conversion on the back end) given some datastructure that doesn't easily convert, it would be harder to convert to csv from the json.

I'll think more about this.

vsoch commented 6 years ago

For now, try this:

create the experiment and don't think about "how it's saved" - get something running and ping me so I can take a look
Then we determine some good ideas for getting whatever is generated in the web interface to csv, ultimately. We need to take into account flexibility to not break depending on different user preferences.

earcanal commented 6 years ago

I'm going to attempt to port so I can run the second part of my study from a container. It's not a showstopper for me if the output is JSON. What's the best way to go about this? Do the port on a branch, test, then merge back into master?

vsoch commented 6 years ago

The general github flow suggests that you should do this work on a new branch, and your master should stay in sync with the upstream master (production).