Difficulty Downloading 2017 HVS Data after Appending New Rows

[ghost]

Hi,

I was trying to add to use lodown to download all HVS data locally, including the new 2017 microtables. I appended new rows with the relevant information for all 3 microdata (occupied, vacant, and persons) but for some reason I am only getting .rds files for the 2017 occupied dataset. After successfully story 19 of 21 catalog entries in the desired location, the code stops with the below error.

[This is my first time on github, so apologies in advance if I've submitted this issue incorrectly]

Here is my code:

library(devtools)
library(lodown)

nychvs_cat <- get_catalog("nychvs", output_dir = file.path( path.expand( "C:\\My Data Sources\\NYCHVS_microdata" ) , "NYCHVS" ) )

nychvs_cat <- rbind(nychvs_cat, c("occ",2017, "https://www2.census.gov/programs-surveys/nychvs/datasets/2017/microdata/uf_17_occ_web_b.txt", "C:/Users/UNHP2010/Documents/R/win-library/3.5/lodown/extdata/nychvs/furman/sas_import_program_2017.txt", 9 ,"C:\\My Data Sources\\NYCHVS_microdata/NYCHVS/2017/occ.rds"))

nychvs_cat <- rbind(nychvs_cat, c("vac",2017, "https://www2.census.gov/programs-surveys/nychvs/datasets/2014/microdata/uf_14_vac_web.dat", "C:/Users/UNHP2010/Documents/R/win-library/3.5/lodown/extdata/nychvs/furman/sas_import_program_2017.txt", 516 ,"C:\\My Data Sources\\NYCHVS_microdata/NYCHVS/2017/vac.rds"))

nychvs_cat <- rbind(nychvs_cat, c("per",2017, "https://www2.census.gov/programs-surveys/nychvs/datasets/2014/microdata/uf_14_pers_web_b.txt", "C:/Users/UNHP2010/Documents/R/win-library/3.5/lodown/extdata/nychvs/furman/sas_import_program_2017.txt", 413 ,"C:\\My Data Sources\\NYCHVS_microdata/NYCHVS/2017/pers.rds"))

nychvs_cat <- lodown("nychvs", nychvs_cat)

The Error: lodown is now exiting unexpectedly. websites that host publicly-downloadable microdata change often and sometimes those changes cause this software to break. if the error call stack below appears to be a hiccup in your internet connection, then please verify your connectivity and retry the download. otherwise, please open a new issue at https://github.com/ajdamico/asdfree/issues with the contents of this error call stack and also the output of your sessionInfo().

[[1]] lodown("nychvs", nychvs_cat)

[[2]] withCallingHandlers(catalog <- load_fun(data_name = data_name, catalog, ...), error = function(e) { print(sessionInfo()) if (grepl("cannot allocate vector of size", e)) message(memory_note) else if (grepl("parameter must be specified", e)) message(parameter_note) else if (grepl("to install", e)) message(installation_note) else { message(unknown_error_note) print(sys.calls()) } })

[[3]] load_fun(data_name = data_name, catalog, ...)

[[4]] read_SAScii(tf, cleaned.sas.script, beginline = catalog[i, "beginline"])

[[5]] suppressWarnings(sasc <- SAScii::parse.SAScii(tf, beginline = beginline, lrecl = lrecl))

[[6]] withCallingHandlers(expr, warning = function(w) invokeRestart("muffleWarning"))

[[7]] SAScii::parse.SAScii(tf, beginline = beginline, lrecl = lrecl)

[[8]] .handleSimpleError(function (e) { print(sessionInfo()) if (grepl("cannot allocate vector of size", e)) message(memory_note) else if (grepl("parameter must be specified", e)) message(parameter_note) else if (grepl("to install", e)) message(installation_note) else { message(unknown_error_note) print(sys.calls()) } }, "missing value where TRUE/FALSE needed", quote(if (x[j - 1, "start"] + x[j - 1, "width"] < start.point) { x[j, "width"] <- (x[j - 1, "start"] + x[j - 1, "width"]) - start.point j <- j + 1 }))

[[9]] h(simpleError(msg, call))

Error in if (x[j - 1, "start"] + x[j - 1, "width"] < start.point) { : missing value where TRUE/FALSE needed

[abesolberg]

Hi. I'm having a similar problem. I updated the get_catalog_nychvs function to include the 2017 data (see code here: https://github.com/abesolberg/lodown/blob/master/R/nychvs.R), however when I try to use the lowdown() function to get the new tables, I get the following error:

[[1]] lodown("nychvs", nychvs2017)

[[2]] withCallingHandlers(catalog <- load_fun(data_name = data_name, catalog, ...), error = function(e) { print(sessionInfo()) if (grepl("cannot allocate vector of size", e)) message(memory_note) else if (grepl("parameter must be specified", e)) message(parameter_note) else if (grepl("to install", e)) message(installation_note) else { message(unknown_error_note) print(sys.calls()) } })

[[3]] load_fun(data_name = data_name, catalog, ...)

[[4]] read_SAScii(tf, cleaned.sas.script, beginline = catalog[i, "beginline"])

[[5]] suppressWarnings(sasc <- SAScii::parse.SAScii(tf, beginline = beginline, lrecl = lrecl))

[[6]] withCallingHandlers(expr, warning = function(w) invokeRestart("muffleWarning"))

[[7]] SAScii::parse.SAScii(tf, beginline = beginline, lrecl = lrecl)

[[8]] SAS.uncomment(SASinput, "*", ";")

[[9]] sub(substr(SASinput[i], slash_asterisk[1], asterisk_slash[1] + 1), "", SASinput[i], fixed = T)

[[10]] .handleSimpleError(function (e) { print(sessionInfo()) if (grepl("cannot allocate vector of size", e)) message(memory_note) else if (grepl("parameter must be specified", e)) message(parameter_note) else if (grepl("to install", e)) message(installation_note) else { message(unknown_error_note) print(sys.calls()) } }, "zero-length pattern", quote(sub(substr(SASinput[i], slash_asterisk[1], asterisk_slash[1] + 1), "", SASinput[i], fixed = T)))

[[11]] h(simpleError(msg, call))

Error in sub(substr(SASinput[i], slash_asterisk[1], asterisk_slash[1] + : zero-length pattern

[ajdamico]

hi! apologies for the long delay. i've made a couple of big updates to asdfree.com that hopefully make the website a bit better, but i've decided to stop maintaining the lodown package so probably won't fix the bug you've reported. the new asdfree does have nychvs data, but only for the most current year. thanks