search

International-Soil-Radiocarbon-Database / ISRaD

Repository for the development and release of ISRaD data and tools
https://international-soil-radiocarbon-database.github.io/ISRaD/
24 stars 15 forks source link

QAQC compile doi T/F issue #56

Closed jb388 closed 5 years ago

jb388 commented 5 years ago

@alkalifly @ShaneStoner @greymonroe @aahoyt Just moving this email conversation here.

The error message is: Error in if ((!(RCurl::url.exists(paste0("https://www.doi.org/", dois[d])) | : missing value where TRUE/FALSE needed

Interestingly, the QAQC works on stand-alone files without throwing this error.

jb388 commented 5 years ago

Just a thought, but are we trimming the header rows twice when we call 'QAQC' from within 'compile'? I suspect this is creating "NA"s that are causing the above error.

ShaneStoner commented 5 years ago

Just a thought, but are we trimming the header rows twice when we call 'QAQC' from within 'compile'? I suspect this is creating "NA"s that are causing the above error.

Are you referring to my last update to the QAQC function? I personally didn't add any clipping, just added 3 to the row number reported by the function so that it matches what you see when you open the excel file, i.e. the function will say "error on row 40" when the error is actually on the .xlsx row 43". Nothing too special.

I noticed the same thing about single files passing but failing during compile. This will have to be a job for someone other than me, I'm not sure how it's working differently between the two files at the moment. But since everything passes QAQC singularly, I think it is still okay that I blocked the QAQC during the compile in the dev branch.

ShaneStoner commented 5 years ago

Update: after looking at the code some more, I found that it runs QAQC on every file individually, then, at the end, attempts to run QAQC on the compiled ISRaD list file. That makes more sense that the error occurs when it does, right at the end after the main compilation processes have been completed. This is the instance of QAQC that I have commented out. I'm not sure what this was supposed to accomplish but I'd be curious to know! It isn't in the master branch version of that function.

jb388 commented 5 years ago

@ShaneStoner No, I meant the other lines that actually trim the header rows (e.g. ln95 in QAQC). But I think the only lines that trim the headers in compile only look at the template file (ln76).

Edit: Ah, yes, maybe this is the problem then. When QAQC is called the second time it tries to trim the description rows again, which creates NAs in the doi field.

greymonroe commented 5 years ago

looks like compile no longer runs QAQC so i will close this