Unable to execute statement 'create table acs2013_1yr as select 'M' as rt, a.serialno, a.division, a.puma, a.region, a.st, a.adjh...'.

[jeffrosenblum]

new error. It made it through the first (h) and crashed somewhere in the second (p). I'm still using the development version of MonetDBLite_0.4.1 so i'll try it with the CRAN version. Though it's possible it's a hard drive space issue, as the MonetDB folder is now 17GB and there is only 6GB left on the HD.

Error in .local(conn, statement, ...) : 
  Unable to execute statement 'create table acs2013_1yr as select 'M' as rt, a.serialno, a.division, a.puma, a.region, a.st, a.adjh...'.
 !ERROR: HEAPextend: failed to extend to 25324760 for 51\5177.tail: h->storage == STORE_MEM && can_map && fd >= 0 && HEAPload_intern() != GDK_SUCCEED '.

[[1]]
lodown("acs", acs_cat)

[[2]]
withCallingHandlers(catalog <- load_fun(data_name = data_name, 
    catalog, ...), error = function(e) {
    if (grepl("cannot allocate vector of size", e)) 
        message(memory_note)
    else if (grepl("parameter must be specified", e)) 
        message(parameter_note)
    else if (grepl("to install", e)) 
        message(installation_note)
    else {
        message(unknown_error_note)
        print(sys.calls())
    }
})

[[3]]
load_fun(data_name = data_name, catalog, ...)

[[4]]
DBI::dbSendQuery(db, i.j)

[[5]]
DBI::dbSendQuery(db, i.j)

[[6]]
.local(conn, statement, ...)

[[7]]
stop("Unable to execute statement '", statement, "'.\nServer says '", 
    env$message, "'.")

[[8]]
.handleSimpleError(function (e) 
{
    if (grepl("cannot allocate vector of size", e)) 
        message(memory_note)
    else if (grepl("parameter must be specified", e)) 
        message(parameter_note)
    else if (grepl("to install", e)) 
        message(installation_note)
    else {
        message(unknown_error_note)
        print(sys.calls())
    }
}, "Unable to execute statement 'create table acs2013_1yr as select 'M' as rt, a.serialno, a.division, a.puma, a.region, a.st, a.adjh...'.\nServer says 'GDKerror:!ERROR: GDKextendf: could not extend file !OS: The printer is out of paper.\r !ERROR: GDKextendf: could not extend file !OS: The printer is out of paper.\r !ERROR: GDKextendf: could not extend file !OS: The printer is out of paper.\r !ERROR: HEAPextend: failed to extend to 25324760 for 51\\5177.tail: h->storage == STORE_MEM && can_map && fd >= 0 && HEAPload_intern() != GDK_SUCCEED '.", 
    quote(.local(conn, statement, ...)))

[[9]]
h(simpleError(msg, call))

> sessionInfo() 
R version 3.4.2 (2017-09-28)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252 
[2] LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] lodown_0.1.0

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.13      knitr_1.17        bindr_0.1         xml2_1.1.1       
 [5] magrittr_1.5      hms_0.3           rvest_0.3.2       archive_1.0.0    
 [9] R6_2.2.2          rlang_0.1.2       stringr_1.2.0     httr_1.3.1       
[13] dplyr_0.7.4       tools_3.4.2       DBI_0.7           selectr_0.3-1    
[17] digest_0.6.12     assertthat_0.2.0  tibble_1.3.4      bindrcpp_0.2     
[21] readr_1.1.1       codetools_0.2-15  curl_3.0          glue_1.1.1.9000  
[25] MonetDBLite_0.4.1 haven_1.1.0       stringi_1.1.5     compiler_3.4.2   
[29] forcats_0.2.0     XML_3.98-1.9      pkgconfig_2.0.1  

[hannes]

Almost certainly a disk space issue.

[ajdamico]

hi, if it doesn't work in dev it's very unlikely it will work on the cran version.. @hannesmuehleisen 6gb of free space seems like a lot? @jeffrosenblum are you using a networked drive or is possibly your C drive full even if the output_dir= folder still has space?

[jeffrosenblum]

Using C: drive. I'll try it using a USB drive with over 150GB of space, changing the output_dir = "D:/ACS/MonetDB"

[hannes]

great thanks

[jeffrosenblum]

Ok, so it looks like the above error was from a HD space constraint issue.

It was all going so well, but I just returned to my computer and there is an "R Session Aborted." Earlier I was monitoring the disk space on the D drive, and the MonetDB folder grew significantly to over 25GB. I checked a while later, and it was down to 12GB. And now after the crash it is 12GB. I don't have any error log to share because of the nature of the problem. It did generate a acs2013_1yr.rds file that is 440MB. The whole process ran for 2 hours it seems, looking at the last time something was written to the MonetDB folder.

Any thoughts? One is to just restart windows and Rstudio and try again. Or maybe it actually finished, but somehow I doubt that. Limitation of my computer memory? Or try it on another similar machine running windows 8?

[jeffrosenblum]

@hannesmuehleisen if you have a suggestion as to whether windows 10 or 8 is better to try, or minimum memory, or some other configuration item, let me know. (You'll probably suggest using linux...)

[hannes]

Great it got further. The issue you are describing now may be related to something I am currently trying to resolve. If you have a couple of days I would suggest to wait a bit. Sorry for that.

[ajdamico]

hi, one other possible workaround is reverting to the last stable version until we knock this crash out.. sorry for the hassle

url <- "http://cran.r-project.org/src/contrib/Archive/MonetDBLite/MonetDBLite_0.3.1.tar.gz"
pkgFile <- file.path( tempdir() , "MonetDBLite_0.3.1.tar.gz" )
download.file(url = url, destfile = pkgFile)
install.packages(pkgs=pkgFile, type="source", repos=NULL)

[jeffrosenblum]

Run on Windows 10, didn't crash but exited with memory error, see below. I'll try that? But wanted to let you know what I got. I was monitoring disk space, and that does not seem to be a problem (destination drive has 150GB, C drive has 17GB. I'll also try using prev version.

R version 3.4.2 (2017-09-28)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] lodown_0.1.0

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.13      survey_3.32-1     xml2_1.1.1        magrittr_1.5     
 [5] splines_3.4.2     hms_0.3           rvest_0.3.2       lattice_0.20-35  
 [9] archive_1.0.0     R6_2.2.2          rlang_0.1.2       stringr_1.2.0    
[13] httr_1.3.1        tools_3.4.2       grid_3.4.2        DBI_0.7          
[17] selectr_0.3-1     survival_2.41-3   digest_0.6.12     tibble_1.3.4     
[21] Matrix_1.2-11     readr_1.1.1       codetools_0.2-15  curl_3.0         
[25] glue_1.1.1.9000   MonetDBLite_0.4.1 haven_1.1.0       stringi_1.1.5    
[29] compiler_3.4.2    forcats_0.2.0     XML_3.98-1.9     

lodown is now exiting due to a memory error.
windows users: your computing performance would suffer due to disk paging,
but you can increase your memory limits with beyond your available hardware with the `?memory.limit` function.
for example, you can set the memory ceiling of an R session to 256 GB by typing `memory.limit(256000)`.

Error: cannot allocate vector of size 1.9 Gb

[ajdamico]

you could try raising the memory ceiling at the start of your session? last line of the error message..

On Thu, Oct 19, 2017 at 7:42 AM, jeffrosenblum notifications@github.com wrote:

Run on Windows 10, didn't crash but exited with memory error, see below. I'll try that? But wanted to let you know what I got. I was monitoring disk space, and that does not seem to be a problem (destination drive has 150GB, C drive has 17GB. I'll also try using prev version.

R version 3.4.2 (2017-09-28) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 [3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C [5] LC_TIME=English_United States.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] lodown_0.1.0

loaded via a namespace (and not attached): [1] Rcpp_0.12.13 survey_3.32-1 xml2_1.1.1 magrittr_1.5 [5] splines_3.4.2 hms_0.3 rvest_0.3.2 lattice_0.20-35 [9] archive_1.0.0 R6_2.2.2 rlang_0.1.2 stringr_1.2.0 [13] httr_1.3.1 tools_3.4.2 grid_3.4.2 DBI_0.7 [17] selectr_0.3-1 survival_2.41-3 digest_0.6.12 tibble_1.3.4 [21] Matrix_1.2-11 readr_1.1.1 codetools_0.2-15 curl_3.0 [25] glue_1.1.1.9000 MonetDBLite_0.4.1 haven_1.1.0 stringi_1.1.5 [29] compiler_3.4.2 forcats_0.2.0 XML_3.98-1.9

lodown is now exiting due to a memory error. windows users: your computing performance would suffer due to disk paging, but you can increase your memory limits with beyond your available hardware with the ?memory.limit function. for example, you can set the memory ceiling of an R session to 256 GB by typing memory.limit(256000).

Error: cannot allocate vector of size 1.9 Gb

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/ajdamico/asdfree/issues/299#issuecomment-337882363, or mute the thread https://github.com/notifications/unsubscribe-auth/AANO56Fx6KgzZ2PNST_dQUF67xFkfHIPks5stzWegaJpZM4P9rCj .

[jeffrosenblum]

success increasing the memory limit using memory.limit(256000). FYI, worked on a Windows 10 computer with only 4GB memory. Thanks!

acs catalog entry 1 of 1 stored in 'acs2015_1yr'

acs local download completed

[ajdamico]

sorry for the hassle