International-Soil-Radiocarbon-Database / ISRaD

Repository for the development and release of ISRaD data and tools
https://international-soil-radiocarbon-database.github.io/ISRaD/
24 stars 15 forks source link

ISRaD_extra_list.xlsx problem with Inf numerical values #178

Closed alkalifly closed 5 years ago

alkalifly commented 5 years ago

Something is happening when the ISRaD_extra_list.xlsx gets written where all values of Inf or -Inf end up as #NUM! See, for example, the lyr_bot entry for Biedenbender_2004, Walnut Gulch, walnut center, walnut_center186+.

The non-extra ISRaD_list.xlsx file is fine, and the ISRaD_extra_flat_xxx.csv files are also fine, so it is something specific in the writing of extra Excel file.

jb388 commented 5 years ago

Nice sleuthing, @alkalifly I don't believe Excel has a way of recording infinite values: see. Sounds like we will have to decide on an infinity proxy value, e.g. 1e99. Replacing Inf values with the proxy will have to be done before the excel file is output.

greymonroe commented 5 years ago

Another possible solution is to convert everything to character data types before saving which will convert Inf to "Inf" and should save it in the excel file accordingly. Then when R reads the excel file it inteprets "Inf" as Inf automatically so it should be seamless. I will work on implementing and testing.

alkalifly commented 5 years ago

The solution that @greymonroe seems like a good one because the ISRaD_list.xlsx file (non-extra) is already all in character data, which is why it doesn't have this problem. Making ISRaD_extra_list.xlsx also be character data would keep everything consistent. Also, @greymonroe while you are working on this, if it is an easy enough thing to do, it might be good to have ISRaD_extra_list.xlsx file have the same full three-row header (with description and units) as the ISRaD_list.xlsx file does. Currently it only includes the top row, so row numbers are not consistent between the two files (i.e., row 4 in ISRaD_list.xlsx is the same entry as row 2 of ISRaD_estra_list.xlsx)

greymonroe commented 5 years ago

fixed in recent updates