Closed rpruim closed 9 years ago
A little more testing
> res <- Map(CIAdata, name=CIAdata()$Name)
Warning messages:
1: In (function (..., deparse.level = 1) :
number of columns of result is not a multiple of vector length (arg 1)
2: In (function (..., deparse.level = 1) :
number of columns of result is not a multiple of vector length (arg 2)
3: In (function (name = NULL) : NAs introduced by coercion
> length(res)
[1] 75
> sapply(res, nrow) %>% favstats
min Q1 median Q3 max mean sd n missing
3 181 211 219 257 193.0667 43.05412 75 0
> sapply(res, nrow) %>% sort %>% head
inflation abroadStock waterways homeStock HIVdeath shares
3 100 107 111 119 120
Things seem good for all but inflation
:
> CIAdata("inflation")
Retrieving data from https://www.cia.gov/library/publications/the-world-factbook/rankorder/rawdata_2092.txt
country
1 (GEOCODE,CODEDESC,NUMFACT,TEXT,FIELDDESC)
2 Insert into FACTBOOK.RO_2092 (GEOCODE,CODEDESC,NUMFACT,TEXT,FIELDDESC) values ('BK','Bosnia and Herzegovina',' -0.80','2014 est.','Inflation rate (consumer prices)');
3 Insert into FACTBOOK.RO_2092 (GEOCODE,CODEDESC,NUMFACT,TEXT,FIELDDESC) values ('LS','Liechtenstein',' -0.70','2012','Inflation rate (consumer prices)');
inflation
1 NA
2 2
3 NA
Warning messages:
1: In (function (..., deparse.level = 1) :
number of columns of result is not a multiple of vector length (arg 2)
2: In CIAdata("inflation") : NAs introduced by coercion
This seems to be messed up at CIA. The HTML page is bad too.
I'm going to leave this in our index of available data sets for now, in case the data return. But we could just delete that row from CIA.rda
A little debugging leads me to suspect that the file format has changed. We are trying to read files with
which suggests we are expecting tab-delimited files. But the files appear to be fixed width formatted, and I don't know how we get the column widths.
It looks like using a double space as separator would work (at least for this file https://www.cia.gov/library/publications/the-world-factbook/rankorder/rawdata_2001.txt), but the
read.*
functions appear to only allow one-character separators.My current solution is to replace
with
This seems to be working in limited testing
But I don't know with certainty that it will work on all files.