search

JGCRI / gcamdata

The GCAM data system
https://jgcri.github.io/gcamdata/
Other
44 stars 26 forks source link

Error or warning while reading FAO_GDP_Deflators.csv #1254

Closed Tonyzhao610 closed 9 months ago

Tonyzhao610 commented 9 months ago

When I modify the CSV input data in inst, gcamdata gets an error. Running gcamdata with the original csv file does not have such problems. May I ask why making changes in the csv causes gcamdata to run incorrectly? r version 4.3.1 Win11

target common.FAO_GDP_Deflators Error : Unknown shortcut: , In addition: Warning messages: 1: Auto-saved .RData file detected. Remove it to enhance reproducibility. 2: In extract_header_info(header, label = "Column types:", fqfn, required = TRUE) : Multiple commas at end of header line in C:/Users/Admin/AppData/Local/R/cache/R/renv/library/gcamdata-2a9619ea/R-4.3/x86_64-w64-mingw32/gcamdata/extdata/common/FAO_GDP_Deflators.csv ✖ fail common.FAO_GDP_Deflators Error: target common.FAO_GDP_Deflators failed. diagnose(common.FAO_GDP_Deflators)$error$message: Error or warning while reading FAO_GDP_Deflators.csv diagnose(common.FAO_GDP_Deflators)$error$calls: load_csv_files("common/FAO_GDP_Deflators", FALSE, quiet = TRUE, dummy = file_in("C:/Users/Admin/AppData/Local/R/cache/R/renv/library/gcamdata-2a9619ea/R-4.3/x86_64-w64-mingw32/gcamdata/extdata/common/FAO_GDP_Deflators.csv")) stop("Error or warning while reading ", basename(fqfn))

pkyle commented 9 months ago

It looks like you saved the CSV file in Excel, so you just need to open it up in a text editor and (1) remove all trailing commas at the end of each header row, and (2) if any of the header rows have been surrounded by quotation marks, remove those quotation marks. When a CSV is saved in Excel, each row is assigned the same number of commas, based on the row with the most columns in the data table, so header rows (which typically only have one cell with data) get a bunch of commas added to the end of the text. For version control purposes these trailing commas cause an error in gcamdata. Also, if a comment has a comma within a cell in Excel, then all text within the cell will be placed into quotes, which can result in a quotation mark " before the comment character # which causes errors in R is reading the data table.

Tonyzhao610 commented 9 months ago

@pkyle The problem has been solved, thank you so much, this problem has been bothering me for a long time!