Come up with system for indicating which studies are ready to process

[teixeirak]

@ValentineHerr (and @mawilliams99 ), we need a good system to communicate which studies we want to send into the pipeline for sending to EFDB (i.e., generate files to review). This is step 7 here).

For the first round, I sent Valentine a list (see issue #1), and some of those (not necessarily high priority) remain in files to review. (@ValentineHerr , I think we should just delete these. They still need various levels of review, and aren't necessarily high priority.)

@ValentineHerr , from a quick look, I didn't see in your code where you entered the studies identified. What would be a convenient way to communicate these? Perhaps add a column in citations.csv indicating EFDB status (ready to run & send, sent...)? Or have such a file here?

[teixeirak]

note: I think it's more productive to do most of the review in the original ForC format, and only send to 1-to-review once we're confident that it's good. Perhaps a good workflow to start would be for Madison to check data in ForC against the original pub, and then I review/approve the files in 1-to-review.

[teixeirak]

For now, I'll start a list here of studies that are ready:

• [ ] McGarvey_2015_csio

[ValentineHerr]

@ValentineHerr , from a quick look, I didn't see in your code where you entered the studies identified. What would be a convenient way to communicate these? Perhaps add a column in citations.csv indicating EFDB status (ready to run & send, sent...)? Or have such a file here?

I had hard-coded the list of citations that you had listed here.

I think that either a new column in the CITATIONS table or a new .csv file (only one column for citation.ID suffices) in this repo would be fine. I don't think we need to keep track of status as this folder should list all what has been transferred.

Also, it does not matter if a citation that has already been transferred stays in the list as the system makes it that if there is no new data entered (or approved, e.g. if it was listed as suspicious and is not anymore) from that citation, nothing will be generated.

[ValentineHerr]

note: I think it's more productive to do most of the review in the original ForC format, and only send to 1-to-review once we're confident that it's good. Perhaps a good workflow to start would be for Madison to check data in ForC against the original pub, and then I review/approve the files in 1-to-review.

That is fine with me.

[teixeirak]

I added EFDB.ready field to the citations table metadata. @mawilliams99 , could you please add this to ForC_citations when you have a chance? Then, you can populate with "0" for all studies but those in this folder, which get "1". McGarvey_2015_csio should also get "1", as should any the studies you finish reviewing.

Then, @ValentineHerr , you can have your script reference this field.

[ValentineHerr]

Hopefully I did not create a conflict by doing this (I reffered to the wrong issue but the commit is here). If i did, @mawilliams99, you can revert my commit

[teixeirak]

I think we likely have a conflict with @mawilliams99's version (at least the measurements file) after you updated everything, @ValentineHerr , and after I fixed the one error. Sorry, @mawilliams99! The easiest fix is probably to save your copy under a temporary name, pull the new version, and copy your changes in there. Alternatively, sometimes GitHub may be able to merge conflicting copies when the changes themselves don't conflict (have seen it work for .Rmd files, but not sure about .csv).

[ValentineHerr]

I added source.type (but didn't fill) in CITATIONS so you don't need to do it @mawilliams99, unless you have a conflict and need to revert my commit.

If you are having issues with merging your changes in MEASUREMENTS, let me know and I can help you navigate that.

[ValentineHerr]

FYI, @teixeirak McGarvey_2015_csio is ready to review here

[teixeirak]

I just approved McGarvey_2015_csio. I believe we can close this issue.