HMDA failed to load into DB correctly

[plnnr]

I tried to download the data, but this is the error I got:

`> lodown( "hmda" , output_dir = file.path( path.expand( "~" ) , "HMDA" ) ) building catalog for hmda

locally downloading hmda

'https://www.ffiec.gov/hmdarawdata/LAR/National/2006HMDAlar%20-%20National.zip' cached in 'C:/Users/nkobel/AppData/Local/Temp/1/a9a85c9cfea46b46da0bc6f4598d130c.Rcache' copying to 'C:\Users\nkobel\AppData\Local\Temp\1\Rtmps728v9\file404520a4b9f'

R version 3.5.0 (2018-04-23) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C LC_TIME=English_United States.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] lodown_0.1.0

loaded via a namespace (and not attached): [1] Rcpp_0.12.17 digest_0.6.15 withr_2.1.2 R6_2.2.2 DBI_1.0.0 git2r_0.21.0 magrittr_1.5.0 [8] RSQLite_2.1.1 httr_1.3.1 stringi_1.1.7 curl_3.2 blob_1.1.1 SAScii_1.0 xml2_1.2.0
[15] devtools_1.13.5 tools_3.5.0 bit64_0.9-7 stringr_1.3.1 selectr_0.4-1 bit_1.1-14 yaml_2.1.19
[22] compiler_3.5.0 pkgconfig_2.0.1 rvest_0.3.2 memoise_1.1.0

lodown is now exiting unexpectedly. websites that host publicly-downloadable microdata change often and sometimes those changes cause this software to break. if the error call stack below appears to be a hiccup in your internet connection, then please verify your connectivity and retry the download. otherwise, please open a new issue at https://github.com/ajdamico/asdfree/issues with the contents of this error call stack and also the output of your sessionInfo().

[[1]] lodown("hmda", output_dir = file.path(path.expand("~"), "HMDA"))

[[2]] withCallingHandlers(catalog <- load_fun(data_name = data_name, catalog, ...), error = function(e) { print(sessionInfo()) if (grepl("cannot allocate vector of size", e)) message(memory_note) else if (grepl("parameter must be specified", e)) message(parameter_note) else if (grepl("to install", e)) message(installation_note) else { message(unknown_error_note) print(sys.calls()) } })

[[3]] load_fun(data_name = data_name, catalog, ...)

[[4]] DBI::dbSendQuery(db, sql.copy)

[[5]] DBI::dbSendQuery(db, sql.copy)

[[6]] .local(conn, statement, ...)

[[7]] new("SQLiteResult", sql = statement, ptr = result_create(conn@ptr, statement), conn = conn, bigint = conn@bigint)

[[8]] initialize(value, ...)

[[9]] initialize(value, ...)

[[10]] result_create(conn@ptr, statement)

[[11]] stop(list(message = "near \"copy\": syntax error", call = result_create(conn@ptr, statement), cppstack = list(file = "", line = -1L, stack = "C++ stack not available on this system")))

[[12]] (function (e) { print(sessionInfo()) if (grepl("cannot allocate vector of size", e)) message(memory_note) else if (grepl("parameter must be specified", e)) message(parameter_note) else if (grepl("to install", e)) message(installation_note) else { message(unknown_error_note) print(sys.calls()) } })(list(message = "near \"copy\": syntax error", call = result_create(conn@ptr, statement), cppstack = list(file = "", line = -1L, stack = "C++ stack not available on this system")))

Error in result_create(conn@ptr, statement) : near "copy": syntax error In addition: Warning message: Closing open result set, pending rows year type full_url merge_table 1 2006 hmda_lar https://www.ffiec.gov/hmdarawdata/LAR/National/2006HMDAlar%20-%20National.zip hmda_2006 12 2006 pmic_lar https://www.ffiec.gov/pmicrawdata/LAR/National/2006PMIClar%20-%20National.zip pmic_2006 20 2006 hmda_inst https://www.ffiec.gov/hmdarawdata/OTHER/2006HMDAinstitutionrecords.zip hmda_2006 31 2006 pmic_inst https://www.ffiec.gov/pmicrawdata/OTHER/2006PMICinstitutionrecords.zip pmic_2006 2 2007 hmda_lar https://www.ffiec.gov/hmdarawdata/LAR/National/2007HMDAlar%20-%20National.zip hmda_2007 13 2007 pmic_lar https://www.ffiec.gov/pmicrawdata/LAR/National/2007PMIClar%20-%20National.zip pmic_2007 21 2007 hmda_inst https://www.ffiec.gov/hmdarawdata/OTHER/2007HMDAinstitutionrecords.zip hmda_2007 32 2007 pmic_inst https://www.ffiec.gov/pmicrawdata/OTHER/2007PMICinstitutionrecords.zip pmic_2007 39 2007 hmda_reporter https://www.ffiec.gov/hmdarawdata/OTHER/2007HMDAReporterPanel.zip 49 2007 pmic_reporter https://www.ffiec.gov/pmicrawdata/OTHER/2007PMICReporterPanel.zip 56 2007 hmda_msa https://www.ffiec.gov/hmdarawdata/OTHER/2007HMDAMSAOffice.zip 66 2007 pmic_msa https://www.ffiec.gov/hmdarawdata/OTHER/2007HMDAMSAOffice.zip 3 2008 hmda_lar https://www.ffiec.gov/hmdarawdata/LAR/National/2008HMDAlar%20-%20National.zip hmda_2008 14 2008 pmic_lar https://www.ffiec.gov/pmicrawdata/LAR/National/2008PMIClar%20-%20National.zip pmic_2008 22 2008 hmda_inst https://www.ffiec.gov/hmdarawdata/OTHER/2008HMDAinstitutionrecords.zip hmda_2008 33 2008 pmic_inst https://www.ffiec.gov/pmicrawdata/OTHER/2008PMICinstitutionrecords.zip pmic_2008 40 2008 hmda_reporter https://www.ffiec.gov/hmdarawdata/OTHER/2008HMDAReporterPanel.zip 50 2008 pmic_reporter https://www.ffiec.gov/pmicrawdata/OTHER/2008PMICReporterPanel.zip 57 2008 hmda_msa https://www.ffiec.gov/hmdarawdata/OTHER/2008HMDAMSAOffice.zip 67 2008 pmic_msa https://www.ffiec.gov/hmdarawdata/OTHER/2008HMDAMSAOffice.zip 4 2009 hmda_lar https://www.ffiec.gov/hmdarawdata/LAR/National/2009HMDAlar%20-%20National.zip hmda_2009 15 2009 pmic_lar https://www.ffiec.gov/pmicrawdata/LAR/National/2009PMIClar%20-%20National.zip pmic_2009 23 2009 hmda_inst https://www.ffiec.gov/hmdarawdata/OTHER/2009HMDAinstitutionrecords.zip hmda_2009 34 2009 pmic_inst https://www.ffiec.gov/pmicrawdata/OTHER/2009PMICinstitutionrecords.zip pmic_2009 41 2009 hmda_reporter https://www.ffiec.gov/hmdarawdata/OTHER/2009HMDAReporterPanel.zip 51 2009 pmic_reporter https://www.ffiec.gov/pmicrawdata/OTHER/2009PMICReporterPanel.zip 58 2009 hmda_msa https://www.ffiec.gov/hmdarawdata/OTHER/2009HMDAMSAOffice.zip 68 2009 pmic_msa https://www.ffiec.gov/hmdarawdata/OTHER/2009HMDAMSAOffice.zip 5 2010 hmda_lar https://www.ffiec.gov/hmdarawdata/LAR/National/2010HMDAlar%20-%20National.zip hmda_2010 16 2010 pmic_lar https://www.ffiec.gov/pmicrawdata/LAR/National/2010PMIClar%20-%20National.zip pmic_2010 24 2010 hmda_inst https://www.ffiec.gov/hmdarawdata/OTHER/2010HMDAinstitutionrecords.zip hmda_2010 35 2010 pmic_inst https://www.ffiec.gov/pmicrawdata/OTHER/2010PMICinstitutionrecords.zip pmic_2010 42 2010 hmda_reporter https://www.ffiec.gov/hmdarawdata/OTHER/2010HMDAReporterPanel.zip 52 2010 pmic_reporter https://www.ffiec.gov/pmicrawdata/OTHER/2010PMICReporterPanel.zip 59 2010 hmda_msa https://www.ffiec.gov/hmdarawdata/OTHER/2010HMDAMSAOffice.zip 69 2010 pmic_msa https://www.ffiec.gov/hmdarawdata/OTHER/2010HMDAMSAOffice.zip 6 2011 hmda_lar https://www.ffiec.gov/hmdarawdata/LAR/National/2011HMDAlar%20-%20National.zip hmda_2011 17 2011 pmic_lar https://www.ffiec.gov/pmicrawdata/LAR/National/2011PMIClar%20-%20National.zip pmic_2011 25 2011 hmda_inst https://www.ffiec.gov/hmdarawdata/OTHER/2011HMDAinstitutionrecords.zip hmda_2011 36 2011 pmic_inst https://www.ffiec.gov/pmicrawdata/OTHER/2011PMICinstitutionrecords.zip pmic_2011 43 2011 hmda_reporter https://www.ffiec.gov/hmdarawdata/OTHER/2011HMDAReporterPanel.zip 53 2011 pmic_reporter https://www.ffiec.gov/pmicrawdata/OTHER/2011PMICReporterPanel.zip 60 2011 hmda_msa https://www.ffiec.gov/hmdarawdata/OTHER/2011HMDAMSAOffice.zip 70 2011 pmic_msa https://www.ffiec.gov/hmdarawdata/OTHER/2011HMDAMSAOffice.zip 7 2012 hmda_lar https://www.ffiec.gov/hmdarawdata/LAR/National/2012HMDAlar%20-%20National.zip hmda_2012 18 2012 pmic_lar https://www.ffiec.gov/pmicrawdata/LAR/National/2012PMIClar%20-%20National.zip pmic_2012 26 2012 hmda_inst https://www.ffiec.gov/hmdarawdata/OTHER/2012HMDAinstitutionrecords.zip hmda_2012 37 2012 pmic_inst https://www.ffiec.gov/pmicrawdata/OTHER/2012PMICinstitutionrecords.zip pmic_2012 44 2012 hmda_reporter https://www.ffiec.gov/hmdarawdata/OTHER/2012HMDAReporterPanel.zip 54 2012 pmic_reporter https://www.ffiec.gov/pmicrawdata/OTHER/2012PMICReporterPanel.zip 61 2012 hmda_msa https://www.ffiec.gov/hmdarawdata/OTHER/2012HMDAMSAOffice.zip 71 2012 pmic_msa https://www.ffiec.gov/hmdarawdata/OTHER/2012HMDAMSAOffice.zip 8 2013 hmda_lar https://www.ffiec.gov/hmdarawdata/LAR/National/2013HMDAlar%20-%20National.zip hmda_2013 19 2013 pmic_lar https://www.ffiec.gov/pmicrawdata/LAR/National/2013PMIClar%20-%20National.zip pmic_2013 27 2013 hmda_inst https://www.ffiec.gov/hmdarawdata/OTHER/2013HMDAinstitutionrecords.zip hmda_2013 38 2013 pmic_inst https://www.ffiec.gov/pmicrawdata/OTHER/2013PMICinstitutionrecords.zip pmic_2013 45 2013 hmda_reporter https://www.ffiec.gov/hmdarawdata/OTHER/2013HMDAReporterPanel.zip 55 2013 pmic_reporter https://www.ffiec.gov/pmicrawdata/OTHER/2013PMICReporterPanel.zip 62 2013 hmda_msa https://www.ffiec.gov/hmdarawdata/OTHER/2013HMDAMSAOffice.zip 72 2013 pmic_msa https://www.ffiec.gov/hmdarawdata/OTHER/2013HMDAMSAOffice.zip 9 2014 hmda_lar https://www.ffiec.gov/hmdarawdata/LAR/National/2014HMDAlar%20-%20National.zip hmda_2014 28 2014 hmda_inst https://www.ffiec.gov/hmdarawdata/OTHER/2014HMDAinstitutionrecords.zip hmda_2014 46 2014 hmda_reporter https://www.ffiec.gov/hmdarawdata/OTHER/2014HMDAReporterPanel.zip 63 2014 hmda_msa https://www.ffiec.gov/hmdarawdata/OTHER/2014HMDAMSAOffice.zip 10 2015 hmda_lar https://www.ffiec.gov/hmdarawdata/LAR/National/2015HMDAlar%20-%20National.zip hmda_2015 29 2015 hmda_inst https://www.ffiec.gov/hmdarawdata/OTHER/2015HMDAinstitutionrecords.zip hmda_2015 47 2015 hmda_reporter https://www.ffiec.gov/hmdarawdata/OTHER/2015HMDAReporterPanel.zip 64 2015 hmda_msa https://www.ffiec.gov/hmdarawdata/OTHER/2015HMDAMSAOffice.zip 11 2016 hmda_lar https://www.ffiec.gov/hmdarawdata/LAR/National/2016HMDAlar%20-%20National.zip hmda_2016 30 2016 hmda_inst https://www.ffiec.gov/hmdarawdata/OTHER/2016HMDAinstitutionrecords.zip hmda_2016 48 2016 hmda_reporter https://www.ffiec.gov/hmdarawdata/OTHER/2016HMDAReporterPanel.zip 65 2016 hmda_msa https://www.ffiec.gov/hmdarawdata/OTHER/2016HMDAMSAOffice.zip dbfile db_tablename case_count 1 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_lar_2006 NA 12 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_lar_2006 NA 20 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_inst_2006 NA 31 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_inst_2006 NA 2 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_lar_2007 NA 13 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_lar_2007 NA 21 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_inst_2007 NA 32 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_inst_2007 NA 39 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_reporter_2007 NA 49 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_reporter_2007 NA 56 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_msa_2007 NA 66 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_msa_2007 NA 3 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_lar_2008 NA 14 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_lar_2008 NA 22 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_inst_2008 NA 33 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_inst_2008 NA 40 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_reporter_2008 NA 50 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_reporter_2008 NA 57 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_msa_2008 NA 67 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_msa_2008 NA 4 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_lar_2009 NA 15 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_lar_2009 NA 23 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_inst_2009 NA 34 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_inst_2009 NA 41 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_reporter_2009 NA 51 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_reporter_2009 NA 58 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_msa_2009 NA 68 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_msa_2009 NA 5 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_lar_2010 NA 16 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_lar_2010 NA 24 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_inst_2010 NA 35 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_inst_2010 NA 42 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_reporter_2010 NA 52 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_reporter_2010 NA 59 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_msa_2010 NA 69 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_msa_2010 NA 6 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_lar_2011 NA 17 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_lar_2011 NA 25 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_inst_2011 NA 36 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_inst_2011 NA 43 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_reporter_2011 NA 53 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_reporter_2011 NA 60 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_msa_2011 NA 70 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_msa_2011 NA 7 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_lar_2012 NA 18 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_lar_2012 NA 26 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_inst_2012 NA 37 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_inst_2012 NA 44 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_reporter_2012 NA 54 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_reporter_2012 NA 61 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_msa_2012 NA 71 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_msa_2012 NA 8 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_lar_2013 NA 19 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_lar_2013 NA 27 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_inst_2013 NA 38 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_inst_2013 NA 45 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_reporter_2013 NA 55 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_reporter_2013 NA 62 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_msa_2013 NA 72 C:/Users/nkobel/Documents/HMDA/SQLite.db pmic_msa_2013 NA 9 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_lar_2014 NA 28 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_inst_2014 NA 46 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_reporter_2014 NA 63 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_msa_2014 NA 10 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_lar_2015 NA 29 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_inst_2015 NA 47 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_reporter_2015 NA 64 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_msa_2015 NA 11 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_lar_2016 NA 30 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_inst_2016 NA 48 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_reporter_2016 NA 65 C:/Users/nkobel/Documents/HMDA/SQLite.db hmda_msa_2016 NA`

`> sessionInfo() R version 3.5.0 (2018-04-23) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C LC_TIME=English_United States.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] lodown_0.1.0

loaded via a namespace (and not attached): [1] Rcpp_0.12.17 digest_0.6.15 withr_2.1.2 R6_2.2.2 DBI_1.0.0 git2r_0.21.0 magrittr_1.5.0 [8] RSQLite_2.1.1 httr_1.3.1 stringi_1.1.7 curl_3.2 blob_1.1.1 SAScii_1.0 xml2_1.2.0
[15] devtools_1.13.5 tools_3.5.0 bit64_0.9-7 stringr_1.3.1 selectr_0.4-1 bit_1.1-14 yaml_2.1.19
[22] compiler_3.5.0 pkgconfig_2.0.1 rvest_0.3.2 memoise_1.1.0 `

[ajdamico]

yep, sorry. follow #131 for hmda progress, thanks

[plnnr]

I'm not sure if #131 provides a solution to getting the data to load.

[ajdamico]

it's not going to work until it's re-written. you're welcome to contribute, thanks