search

JGCRI / gcamdata

The GCAM data system
https://jgcri.github.io/gcamdata/
Other
44 stars 26 forks source link

driver() failing at module_aglu_LB141.ag_Fert_IFA_ctry_crop... #1232

Closed robbieorvis closed 1 year ago

robbieorvis commented 1 year ago

Four of us are getting the same error running the data system on Windows from the main branch:

[1] "module_aglu_LB141.ag_Fert_IFA_ctry_crop" Error in left_join(): ! ... must be empty. ✖ Problematic argument: • ignore_columns = "item" Run rlang::last_error() to see where the error occurred.

rlang::last_error() <error/rlib_error_dots_nonempty> Error in left_join(): ! ... must be empty. ✖ Problematic argument: • ignore_columns = "item"

Backtrace:

  1. gcamdata::driver()

  2. dplyr:::left_join.data.frame(...) Run rlang::last_trace() to see the full context. rlang::last_trace() <error/rlib_error_dots_nonempty> Error in left_join(): ! ... must be empty. ✖ Problematic argument: • ignore_columns = "item"

    Backtrace: ▆

  3. ├─gcamdata::driver()
  4. │ └─gcamdata:::run_chunk(chunk, all_data[input_names]) at gcam_data_system/R/driver.R:364:6
  5. │ ├─base::do.call(chunk, list(driver.MAKE, all_data)) at gcam_data_system/R/driver.R:16:2
  6. │ └─gcamdata:::module_aglu_LB141.ag_Fert_IFA_ctry_crop("MAKE", <named list>)
  7. │ └─... %>% select(iso, GTAP_crop, scaler) at gcam_data_system/R/zchunk_LB141.ag_Fert_IFA_ctry_crop.R:108:4
  8. ├─dplyr::select(., iso, GTAP_crop, scaler)
  9. ├─dplyr::left_join(...)
  10. └─dplyr:::left_join.data.frame(...)
  11. └─rlang::check_dots_empty0(...)
  12. └─rlang::check_dots_empty(call = call)
  13. └─rlang:::action_dots(...)
  14. ├─base (local) try_dots(...)
  15. └─rlang (local) action(...)
russellhz commented 1 year ago

Hi @robbieorvis - are you using renv? Package versions are often the source of these sort of errors: https://jgcri.github.io/gcamdata/articles/getting-started/getting-started.html#optional-renv-currently-included-in-gcam-6-0-and-gcamdata-release

robbieorvis commented 1 year ago

That failed for all of us when we tried to follow the instructions online. Happy to try it again and share the output here.

russellhz commented 1 year ago

Probably best to get that working. To be clear, you are using the main GCAM branch from the gcam-core repository: https://github.com/JGCRI/gcam-core?

robbieorvis commented 1 year ago

For the data system, no. We've been using the main branch of the GCAM data system repository: https://github.com/JGCRI/gcamdata

russellhz commented 1 year ago

OK, that shouldn't matter, but good to know in case getting renv working doesn't solve the problem

robbieorvis commented 1 year ago

Okay, when trying renv::restore(), I get the following error:

ERROR: compilation failed for package 'curl'

russellhz commented 1 year ago

You can try running devtools::install_version("curl", version = "4.3") -- you will need devtools installed first (install.packages("devtools"))

If you run into problems, can you also share your results when you run sessionInfo()?

robbieorvis commented 1 year ago

Okay, failed again. Here's the latest:

devtools::install_version("curl", version = "4.3") Downloading package from url: https://cloud.r-project.org/src/contrib/Archive/curl/curl_4.3.tar.gz Installing package into ‘C:/Users/robbie/Dropbox (Energy Innovation)/My Documents/GCAM Model FIles/gcam_data_system/renv/library/R-4.2/x86_64-w64-mingw32’ (as ‘lib’ is unspecified) Warning message: Project requested R version '4.1.0' but '4.2.2' is currently being used

  • installing source package 'curl' ... package 'curl' successfully unpacked and MD5 sums checked using staged installation libs rm -f curl.dll callbacks.o curl.o download.o escape.o fetch.o form.o getdate.o handle.o ieproxy.o init.o interrupt.o multi.o nslookup.o reflist.o split.o ssl.o typechecking.o utils.o version.o winidn.o writer.o libwinhttp.dll.a winhttp.def "C:/PROGRA~1/R/R-42~1.2/bin/x64/Rscript.exe" "../tools/winlibs.R" 7.64.1 echo '#include <curl/curl.h>' | gcc -E -I../windows/libcurl-7.64.1/include -DCURL_STATICLIB -DSTRICT_RHEADERS -std=gnu99 -xc - | grep "^[ \t]*CURLOPT.," | sed s/,// > ../tools/option_table.txt cp winhttp64.def.in winhttp.def dlltool --as as -k --dllname winhttp.dll --input-def winhttp.def --output-lib libwinhttp.dll.a gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libcurl-7.64.1/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c callbacks.c -o callbacks.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libcurl-7.64.1/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c curl.c -o curl.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libcurl-7.64.1/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c download.c -o download.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libcurl-7.64.1/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c escape.c -o escape.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libcurl-7.64.1/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c fetch.c -o fetch.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libcurl-7.64.1/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c form.c -o form.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libcurl-7.64.1/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c getdate.c -o getdate.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libcurl-7.64.1/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c handle.c -o handle.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libcurl-7.64.1/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c ieproxy.c -o ieproxy.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libcurl-7.64.1/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c init.c -o init.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libcurl-7.64.1/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c interrupt.c -o interrupt.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libcurl-7.64.1/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c multi.c -o multi.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libcurl-7.64.1/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c nslookup.c -o nslookup.o nslookup.c:17:13: error: conflicting types for 'inet_ntop' 17 | const char inet_ntop(int af, const void src, char dst, socklen_t size); | ^~~~~ In file included from nslookup.c:16: c:\rtools42\x86_64-w64-mingw32.static.posix\include\ws2tcpip.h:401:35: note: previous declaration of 'inet_ntop' was here 401 | WINSOCK_API_LINKAGE LPCSTR WSAAPI InetNtopA(INT Family, LPCVOID pAddr, LPSTR pStringBuf, size_t StringBufSize); | ^~~~~ make: * [C:/PROGRA~1/R/R-42~1.2/etc/x64/Makeconf:253: nslookup.o] Error 1 ERROR: compilation failed for package 'curl'
  • removing 'C:/Users/robbie/Dropbox (Energy Innovation)/My Documents/GCAM Model FIles/gcam_data_system/renv/library/R-4.2/x86_64-w64-mingw32/curl'
  • restoring previous 'C:/Users/robbie/Dropbox (Energy Innovation)/My Documents/GCAM Model FIles/gcam_data_system/renv/library/R-4.2/x86_64-w64-mingw32/curl' Warning message: In i.p(...) : installation of package ‘C:/Users/robbie/AppData/Local/Temp/Rtmpmuqk3W/remotes714425f3467d/curl’ had non-zero exit status sessionInfo() R version 4.2.2 (2022-10-31 ucrt) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.utf8 LC_CTYPE=English_United States.utf8
[3] LC_MONETARY=English_United States.utf8 LC_NUMERIC=C
[5] LC_TIME=English_United States.utf8

attached base packages: [1] stats graphics grDevices datasets utils methods base

other attached packages: [1] gcamdata_5.1 testthat_3.1.6

loaded via a namespace (and not attached): [1] tidyselect_1.2.0 remotes_2.4.2 purrr_1.0.1 vctrs_0.5.2 generics_0.1.3
[6] miniUI_0.1.1.1 htmltools_0.5.4 usethis_2.1.6 utf8_1.2.3 rlang_1.0.6
[11] pkgbuild_1.4.0 later_1.3.0 pillar_1.8.1 urlchecker_1.0.1 glue_1.6.2
[16] withr_2.5.0 sessioninfo_1.2.2 lifecycle_1.0.3 stringr_1.5.0 devtools_2.4.5
[21] htmlwidgets_1.6.1 memoise_2.0.1 callr_3.7.3 fastmap_1.1.0 httpuv_1.6.8
[26] ps_1.7.2 fansi_1.0.4 Rcpp_1.0.10 xtable_1.8-4 renv_0.12.5
[31] promises_1.2.0.1 cachem_1.0.6 pkgload_1.3.2 mime_0.12 fs_1.6.0
[36] brio_1.1.3 digest_0.6.31 stringi_1.7.12 processx_3.8.0 dplyr_1.1.0
[41] shiny_1.7.4 cli_3.6.0 tools_4.2.2 magrittr_2.0.3 tibble_3.1.8
[46] profvis_0.3.7 crayon_1.5.2 tidyr_1.3.0 pkgconfig_2.0.3 ellipsis_0.3.2
[51] data.table_1.14.6 prettyunits_1.1.1 assertthat_0.2.1 rstudioapi_0.14 R6_2.5.1
[56] compiler_4.2.2

pkyle commented 1 year ago

@robbieorvis please delete the argument , ignore_columns = "item" from the line of code in question, which I think is line 138 but might be slightly different depending on your version. It's not an argument of the left_join() function, so while some environments or dplyr versions might allow extraneous arguments without crashing, there's no reason to have it in there. It'll be gone by the time of the next release!

robbieorvis commented 1 year ago

You're talking about the module where the crash is happening using the non renv approach right? That would be this module: module_aglu_LB141.ag_Fert_IFA_ctry_crop

And then I should run it the normal way without renv and just using the normal driver() command?

Just confirming that we are abandoning trying to get renv working and instead trying to debug the R module.

pkyle commented 1 year ago

I don't mean to recommend one way or another whether you should use renv; I've never used it myself. But that one specific line of code within module_aglu_LB141.ag_Fert_IFA_ctry_crop contains an argument that is not recognized by the left_join() function, and dplyr/tidyverse functions often get more stringent over time in terms of what's allowed. If you fix this and then crash later on in the code for a similar reason, and renv isn't working out, then you can use devtools::install_version() to get an older version of dplyr and that should work.

robbieorvis commented 1 year ago

Sorry @pkyle didn't realize that was your response and not one from @russellhz.

That fixed the issue with the previous module, but now that I've made it further, another one popped up:

[1] "module_energy_L210.resources"
Error in mutate() at gcam_data_system/R/utils-data.R:32:2: ℹ In argument: environCost = if_else(resource == "coal" & region %in% L210.low_reg, 0, environCost). Caused by error in if_else(): ! false must be a vector, not NULL. Run rlang::last_error() to see where the error occurred.

rlang::last_error() <error/dplyr:::mutate_error> Error in mutate() at gcam_data_system/R/utils-data.R:32:2: ℹ In argument: environCost = if_else(resource == "coal" & region %in% L210.low_reg, 0, environCost). Caused by error in if_else(): ! false must be a vector, not NULL.

Backtrace:

  1. gcamdata::driver()
  2. dplyr::if_else(resource == "coal" & region %in% L210.low_reg, 0, environCost)

sessionInfo() R version 4.2.2 (2022-10-31 ucrt) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.utf8 LC_CTYPE=English_United States.utf8
[3] LC_MONETARY=English_United States.utf8 LC_NUMERIC=C
[5] LC_TIME=English_United States.utf8

attached base packages: [1] stats graphics grDevices datasets utils methods base

other attached packages: [1] gcamdata_5.1 testthat_3.1.6

loaded via a namespace (and not attached): [1] tidyselect_1.2.0 remotes_2.4.2 purrr_1.0.1 vctrs_0.5.2 generics_0.1.3
[6] miniUI_0.1.1.1 htmltools_0.5.4 usethis_2.1.6 utf8_1.2.3 rlang_1.0.6
[11] pkgbuild_1.4.0 later_1.3.0 pillar_1.8.1 urlchecker_1.0.1 glue_1.6.2
[16] withr_2.5.0 bit64_4.0.5 sessioninfo_1.2.2 lifecycle_1.0.3 stringr_1.5.0
[21] devtools_2.4.5 htmlwidgets_1.6.1 memoise_2.0.1 tzdb_0.3.0 callr_3.7.3
[26] fastmap_1.1.0 httpuv_1.6.8 ps_1.7.2 parallel_4.2.2 fansi_1.0.4
[31] Rcpp_1.0.10 readr_2.1.4 xtable_1.8-4 renv_0.12.5 promises_1.2.0.1 [36] cachem_1.0.6 desc_1.4.2 pkgload_1.3.2 vroom_1.6.1 bit_4.0.5
[41] mime_0.12 fs_1.6.0 brio_1.1.3 hms_1.1.2 digest_0.6.31
[46] stringi_1.7.12 processx_3.8.0 dplyr_1.1.0 shiny_1.7.4 rprojroot_2.0.3
[51] cli_3.6.0 tools_4.2.2 magrittr_2.0.3 tibble_3.1.8 profvis_0.3.7
[56] crayon_1.5.2 tidyr_1.3.0 pkgconfig_2.0.3 ellipsis_0.3.2 data.table_1.14.6 [61] prettyunits_1.1.1 assertthat_0.2.1 rstudioapi_0.14 R6_2.5.1 compiler_4.2.2

pkyle commented 1 year ago

Haha yep, I was worried that was going to happen! When they tighten the rules of functions in tidyverse package updates, we typically have ~5 places that need minor updates to continue to execute. In this case in module_energy_L210.resources this is a bug that was permitted in the older versions of if_else(); to fix it, in that highlighted line of code and the one below, the string environCost should be input.cost. i.e., mutate(environCost = if_else(resource == "coal" & region %in% L210.low_reg, 0, input.cost), environCost = if_else(resource == "coal" & region %in% L210.high_reg, 10 * input.cost, input.cost)) %>% Anyway, unless you'd like to continue walking us through our required modifications for the next dplyr update (I'm not gonna update mine for a while now 😆 ), you probably just want to roll back your local dplyr package to something less stringent. I'm currently using 1.0.10. So, the command to install that one locally would be: devtools::install_version("dplyr", "1.0.10")

robbieorvis commented 1 year ago

Okay, progress! @pkyle rolling back dplyr gets the driver() command working when not using renv. Still can't use _driver_drake().

For @russellhz - there is some issue with curl when running renv::restore(). Curl fails to compile. I can download the binary and install directly in to the renv folder in the gcam-data-system, but when running renv::restore() Rstudio attempts to override and it crashes (I couldn't determine if there is a way to skip trying to restore curl.... as you can probably tell I am an extreme novice with R and RStudio).

Seems like maybe a broader issue than your software with curl generally. All five of us trying to install renv have run into the same issue with curl

russellhz commented 1 year ago

@robbieorvis thanks for catching this - it appears that this is an issue when combining windows with the latest version of R (4.2+) and certain packages (curl, openssl, igraph).

If you would still like to get renv working (which hopefully will also get driver_drake() working), I have created a new renv.lock file for this case. You can simply replace the renv.lock file at the top of your gcamdata directory with this file: https://github.com/JGCRI/gcamdata/blob/renv_lock_R4-2/renv.lock

Then you can try renv::restore() again

robbieorvis commented 1 year ago

Thanks @russellhz. Still getting an issue, this time with rlang. renv::restore() wants to downgrade from 1.0.1 to 0.4.11 but then throws an error that 1.0.1 is needed.

robbieorvis commented 1 year ago

I just modified the lock file to use the newer version of rlang that is already installed and it seems to have worked for that, but then it got caught on xml2. Updated that as well,

Made it all the way to git2r, but that one failed to compile and there doesn't appear to be prior version. See error log below:

Installing git2r [0.29.0] ... FAILED Error installing package 'git2r':

** libs rm -f git2r.dll git2r.o git2r_S3.o git2r_arg.o git2r_blame.o git2r_blob.o git2r_branch.o git2r_checkout.o git2r_clone.o git2r_commit.o git2r_config.o git2r_cred.o git2r_diff.o git2r_error.o git2r_graph.o git2r_index.o git2r_libgit2.o git2r_merge.o git2r_note.o git2r_object.o git2r_odb.o git2r_oid.o git2r_push.o git2r_reference.o git2r_reflog.o git2r_remote.o git2r_repository.o git2r_reset.o git2r_revparse.o git2r_revwalk.o git2r_signature.o git2r_stash.o git2r_status.o git2r_tag.o git2r_transfer.o git2r_tree.o "C:/PROGRA~1/R/R-42~1.2/bin/x64/Rscript.exe" "../tools/winlibs.R" 1.3.0 gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r.c -o git2r.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_S3.c -o git2r_S3.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_arg.c -o git2r_arg.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_blame.c -o git2r_blame.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_blob.c -o git2r_blob.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_branch.c -o git2r_branch.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_checkout.c -o git2r_checkout.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_clone.c -o git2r_clone.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_commit.c -o git2r_commit.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_config.c -o git2r_config.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_cred.c -o git2r_cred.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_diff.c -o git2r_diff.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_error.c -o git2r_error.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_graph.c -o git2r_graph.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_index.c -o git2r_index.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_libgit2.c -o git2r_libgit2.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_merge.c -o git2r_merge.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_note.c -o git2r_note.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_object.c -o git2r_object.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_odb.c -o git2r_odb.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_oid.c -o git2r_oid.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_push.c -o git2r_push.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_reference.c -o git2r_reference.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_reflog.c -o git2r_reflog.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_remote.c -o git2r_remote.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_repository.c -o git2r_repository.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_reset.c -o git2r_reset.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_revparse.c -o git2r_revparse.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_revwalk.c -o git2r_revwalk.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_signature.c -o git2r_signature.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_stash.c -o git2r_stash.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_status.c -o git2r_status.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_tag.c -o git2r_tag.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_transfer.c -o git2r_transfer.o gcc -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I../windows/libgit2-1.3.0/include -DR_NO_REMAP -DSTRICT_R_HEADERS -DGIT2R_HAVE_BUF_DISPOSE -DGIT2R_HAVE_OBJECT_ANY -DGIT2R_LIBGIT2_V0_99_0_RENAMES -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c git2r_tree.c -o git2r_tree.o gcc -shared -s -static-libgcc -o git2r.dll git2r-win.def git2r.o git2r_S3.o git2r_arg.o git2r_blame.o git2r_blob.o git2r_branch.o git2r_checkout.o git2r_clone.o git2r_commit.o git2r_config.o git2r_cred.o git2r_diff.o git2r_error.o git2r_graph.o git2r_index.o git2r_libgit2.o git2r_merge.o git2r_note.o git2r_object.o git2r_odb.o git2r_oid.o git2r_push.o git2r_reference.o git2r_reflog.o git2r_remote.o git2r_repository.o git2r_reset.o git2r_revparse.o git2r_revwalk.o git2r_signature.o git2r_stash.o git2r_status.o git2r_tag.o git2r_transfer.o git2r_tree.o -L../windows/libgit2-1.3.0/lib/x64 -lgit2 -lssh2 -lz -lssl -lcrypto -lwinhttp -lws2_32 -lcrypt32 -lole32 -lrpcrt4 -LC:/rtools42/x86_64-w64-mingw32.static.posix/lib/x64 -LC:/rtools42/x86_64-w64-mingw32.static.posix/lib -LC:/PROGRA~1/R/R-42~1.2/bin/x64 -lR C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: ../windows/libgit2-1.3.0/lib/x64/libgit2.a(diff_print.c.obj):diff_print.c:(.text+0x1342): undefined reference to __imp___iob_func' C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: ../windows/libgit2-1.3.0/lib/x64/libgit2.a(winhttp.c.obj):winhttp.c:(.text+0x1df2): undefined reference toswprintf_s' C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: ../windows/libgit2-1.3.0/lib/x64/libgit2.a(xdiffi.c.obj):xdiffi.c:(.text+0x26a): undefined reference to __imp___iob_func' C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: ../windows/libgit2-1.3.0/lib/x64/libcrypto.a(e_capi.o):(.text+0x359b): undefined reference toimp___iob_func' C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: ../windows/libgit2-1.3.0/lib/x64/libcrypto.a(eng_openssl.o):(.text+0xb): undefined reference to `imp_iob_func' C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: ../windows/libgit2-1.3.0/lib/x64/libcrypto.a(eng_openssl.o):(.text+0x41f): undefined reference to `imp_iob_func' C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: ../windows/libgit2-1.3.0/lib/x64/libcrypto.a(ui_openssl.o):(.text+0x19): undefined reference to `imp_iob_func' C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: ../windows/libgit2-1.3.0/lib/x64/libcrypto.a(ui_openssl.o):(.text+0x757): more undefined references to `imp___iob_func' follow collect2.exe: error: ld returned 1 exit status make: *** [C:/PROGRA~1/R/R-42~1.2/share/make/winshlib.mk:16: git2r.dll] Error 1 ERROR: compilation failed for package 'git2r'

robbieorvis commented 1 year ago

I hit a bunch of bumps along the way, but renv.zip the attached lock file worked at least to get to the point where I could activate renv.

robbieorvis commented 1 year ago

Okay, well driver_drake is running, but now I get the following error:

▶ target gcam.usa.states_subregions Error in loadNamespace(j <- i[[1L]], c(lib.loc, .libPaths()), versionCheck = vI[[j]]) : namespace ‘lifecycle’ 1.0.0 is already loaded, but >= 1.0.3 is required In addition: Warning message: Auto-saved .RData file detected. Remove it to enhance reproducibility. x fail gcam.usa.states_subregions Error: target gcam.usa.states_subregions failed. diagnose(gcam.usa.states_subregions)$error$message: Error or warning while reading states_subregions.csv diagnose(gcam.usa.states_subregions)$error$calls: load_csv_files("gcam-usa/states_subregions", FALSE, quiet = TRUE, dummy = file_in("inst/extdata/gcam-usa/states_subregions.csv")) stop("Error or warning while reading ", basename(fqfn))

russellhz commented 1 year ago

The first error looks like it is still a package issue. It's probably worth trying to install lifecycle version 1.0.3. You may also want to remove that RData file since it looks like drake doesn't like it.

All of that said, if you are only running gcamdata once (i.e. just to run standard GCAM scenarios), using the regular driver should be fine. The advantages of driver_drake mainly come into play if you will be changing the data system.

robbieorvis commented 1 year ago

Okay, finally got this working for the team. The lock file is included here, and here are the steps we had to follow to get it running:

  1. Open the gcamdata.Rproj file
  2. run: install.packages("devtools")
  3. run: devtools::load_all()
  4. run: install.packages( "https://cran.r-project.org/src/contrib/Archive/renv/renv_0.12.5.tar.gz", repos = NULL, type = "source" )
  5. run: renv::init( bare = TRUE)
  6. run: renv::restore()
  7. Go to Session>Restart R
  8. run: devtools::load_all()
  9. run: renv::restore()
  10. run: renv::activate()
  11. run: driver_drake() renv.zip

We are planning to use the data system to create new input data files with some important updates, which is why we went about this in the first place and hence why we are trying to get it working.

I will leave it up to you @russellhz and @pkyle if you want to close this issue at this point, since it's working with the modified lock file.

russellhz commented 1 year ago

I am going to close this - I have added the lockfile that worked for you to #1234