American-Institutes-for-Research / EdSurvey

https://american-institutes-for-research.github.io/EdSurvey/
GNU General Public License v2.0
9 stars 8 forks source link

readPIAAC data from cycle 1 but round 2 or round 3 countries #47

Closed webervienna closed 2 years ago

webervienna commented 2 years ago

I have downloaded the PIAAC data with the suggested function - worked fine for all countries of cycle 1. Reading all countries works without error message, but in fact the countries from the round 2 or round 3 are not loaded properly (e.g. countries='sgp') used readPIAAC('~/PIAAC/Cycle 1/',countries='*')

I have also tried readPIAAC('~/PIAAC/Cycle 1/',countries='sgp'). This works without error message, but there are no data. I hope you can help me figuring out where there could be the issue to solve.

pdbailey0 commented 2 years ago

@webervienna I just downloaded and read the PIAAC data with

downloadPIAAC("~/")
sgp <- readPIAAC('~/PIAAC/Cycle 1/',countries='sgp', force=TRUE)
dim(sgp)
# [1] 5468 1329

When you submit a bug report the form asks you to share your session info. Can you please do that

Here are the instructions again:

  1. Type sessionInfo() in the Rstudio console, and copy and paste the output. This information will help us understand the R environment you are in when encountering the issue.
    # insert the `sessionInfo()` output
    sessionInfo()
webervienna commented 2 years ago

Oh I am sorry I have missed that. Here is the sessionInfo() output:

sessionInfo() R version 4.2.1 (2022-06-23) Platform: x86_64-apple-darwin17.0 (64-bit) Running under: macOS Catalina 10.15.7

Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib

locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] ISOcodes_2022.01.10 EdSurvey_2.7.1 lfactors_1.0.4
[4] car_3.1-0 carData_3.0-5 LaF_0.8.4

loaded via a namespace (and not attached): [1] tidyr_1.2.0 splines_4.2.1 foreach_1.5.2
[4] ellipse_0.4.3 wCorr_1.9.5 Formula_1.2-4
[7] statmod_1.4.37 highr_0.9 cellranger_1.1.0
[10] numDeriv_2016.8-1.1 pillar_1.8.1 backports_1.4.1
[13] lattice_0.20-45 quantreg_5.94 glue_1.6.2
[16] RColorBrewer_1.1-3 minqa_1.2.4 colorspace_2.0-3
[19] Dire_2.1.0 Matrix_1.4-1 plyr_1.8.7
[22] BIFIEsurvey_3.4-15 pkgconfig_2.0.3 pheatmap_1.0.12
[25] broom_1.0.0 SparseM_1.81 haven_2.5.1
[28] xtable_1.8-4 purrr_0.3.4 scales_1.2.1
[31] NPflow_0.13.3 lme4_1.1-30 MatrixModels_0.5-0 [34] tibble_3.1.8 gmp_0.6-6 generics_0.1.3
[37] WeMix_3.2.1 ggplot2_3.3.6 ellipsis_0.3.2
[40] withr_2.5.0 fastcluster_1.2.3 mnormt_2.1.0
[43] Rmpfr_0.8-9 cli_3.3.0 readxl_1.4.1
[46] survival_3.3-1 magrittr_2.0.3 evaluate_0.16
[49] mice_3.14.0 fansi_1.0.3 nlme_3.1-157
[52] MASS_7.3-57 forcats_0.5.2 truncnorm_1.0-8
[55] tools_4.2.1 data.table_1.14.2 hms_1.1.2
[58] mitools_2.4 lifecycle_1.0.1 stringr_1.4.1
[61] munsell_0.5.0 compiler_4.2.1 rlang_1.0.4
[64] grid_4.2.1 nloptr_2.0.3 iterators_1.0.14
[67] NAEPprimer_1.0.1 boot_1.3-28 gtable_0.3.0
[70] codetools_0.2-18 abind_1.4-5 NAEPirtparams_1.0.0 [73] DBI_1.1.3 reshape2_1.4.4 R6_2.5.1
[76] knitr_1.40 dplyr_1.0.9 utf8_1.2.2
[79] miceadds_3.13-12 stringi_1.7.8 Rcpp_1.0.9
[82] vctrs_0.4.1 glm2_1.2.1 xfun_0.32
[85] tidyselect_1.1.2

pdbailey0 commented 2 years ago

Great, and what do you get when you run by readPIAAC call followed by the dim call?

In the bug report the first item is "Brief description of the problem and what output you expect" is there a call that is giving you unexpected output? If so, can you share the call, the output you get, and the output you expect?

webervienna commented 2 years ago

Sorry I will try once more hopefully now in the requested way

BRIEF DESCRIPTION OF ISSUE: ################# I have downloaded the PIAAC data with the suggested function from the OECD webpage - worked fine for all countries of cycle 1. There was no error message. Having a look at the dim I can see that all countries should have some data - that's also what I found when having a look at the plain txt data files. Doing a subset of the countries so I can have a proper look in fact the countries from the round 2 or round 3 are not loaded properly (e.g. countries='sgp'). Dim call returns 0 for number of rows from all round 2 and 3 lines - except for USA 2017 which was not loaded as USA12_14 was included. I would have expected a number different from 0 for those countries of round 2 or 3 (e.g. Singapore). I have tried with traceback call, but this was not supported. Thank you very much for your help.

CALL and OUTPUT FROM R: ######################

rm(list=ls()) library(EdSurvey) Loading required package: car Loading required package: carData Loading required package: lfactors lfactors v1.0.4

EdSurvey v2.7.1

Attaching package: ‘EdSurvey’

The following objects are masked from ‘package:base’:

cbind, rbind

library(ISOcodes)

Download from OECD page

downloadPIAAC('~/') Found downloaded cycle 1 PIAAC file prgautp1.csv. Found downloaded cycle 1 PIAAC file prgbelp1.csv. Found downloaded cycle 1 PIAAC file prgcanp1.csv. Found downloaded cycle 1 PIAAC file prgchlp1.csv. Found downloaded cycle 1 PIAAC file prgczep1.csv. Found downloaded cycle 1 PIAAC file prgdeup1.csv. Found downloaded cycle 1 PIAAC file prgdnkp1.csv. Found downloaded cycle 1 PIAAC file prgecup1.csv. Found downloaded cycle 1 PIAAC file prgespp1.csv. Found downloaded cycle 1 PIAAC file prgestp1.csv. Found downloaded cycle 1 PIAAC file prgfinp1.csv. Found downloaded cycle 1 PIAAC file prgfrap1.csv. Found downloaded cycle 1 PIAAC file prggbrp1.csv. Found downloaded cycle 1 PIAAC file prggrcp1.csv. Found downloaded cycle 1 PIAAC file prghunp1.csv. Found downloaded cycle 1 PIAAC file prgirlp1.csv. Found downloaded cycle 1 PIAAC file prgisrp1.csv. Found downloaded cycle 1 PIAAC file prgitap1.csv. Found downloaded cycle 1 PIAAC file prgjpnp1.csv. Found downloaded cycle 1 PIAAC file prgkazp1.csv. Found downloaded cycle 1 PIAAC file prgkorp1.csv. Found downloaded cycle 1 PIAAC file prgltup1.csv. Found downloaded cycle 1 PIAAC file prgmexp1.csv. Found downloaded cycle 1 PIAAC file prgnldp1.csv. Found downloaded cycle 1 PIAAC file prgnorp1.csv. Found downloaded cycle 1 PIAAC file prgnzlp1.csv. Found downloaded cycle 1 PIAAC file prgperp1.csv. Found downloaded cycle 1 PIAAC file prgpolp1.csv. Found downloaded cycle 1 PIAAC file prgrusp1.csv. Found downloaded cycle 1 PIAAC file prgsgpp1.csv. Found downloaded cycle 1 PIAAC file prgsvkp1.csv. Found downloaded cycle 1 PIAAC file prgsvnp1.csv. Found downloaded cycle 1 PIAAC file prgswep1.csv. Found downloaded cycle 1 PIAAC file prgturp1.csv. Found downloaded cycle 1 PIAAC file CSV_prgusap1.zip. Found downloaded cycle 1 PIAAC file Prgusap1_2017.csv. Found downloaded cycle 1 PIAAC codebook file international-codebook.xlsx.

piaac_all<-readPIAAC('~/PIAAC/Cycle 1/',countries='*') Found cached data for country code “aut”. Found cached data for country code “bel”. Found cached data for country code “can”. Found cached data for country code “chl”. Found cached data for country code “cze”. Found cached data for country code “deu”. Found cached data for country code “dnk”. Found cached data for country code “ecu”. Found cached data for country code “esp”. Found cached data for country code “est”. Found cached data for country code “fin”. Found cached data for country code “fra”. Found cached data for country code “gbr”. Found cached data for country code “grc”. Found cached data for country code “hun”. Found cached data for country code “irl”. Found cached data for country code “isr”. Found cached data for country code “ita”. Found cached data for country code “jpn”. Found cached data for country code “kaz”. Found cached data for country code “kor”. Found cached data for country code “ltu”. Found cached data for country code “mex”. Found cached data for country code “nld”. Found cached data for country code “nor”. Found cached data for country code “nzl”. Found cached data for country code “per”. Found cached data for country code “pol”. Found cached data for country code “rus”. Found cached data for country code “sgp”. Found cached data for country code “svk”. Found cached data for country code “svn”. Found cached data for country code “swe”. Found cached data for country code “tur”. Processing data for country code “usa12_14” . dim(piaac_all) $nrow aut bel can chl cze deu dnk ecu 5130 5463 26683 5212 6102 5465 7328 5702 esp est fin fra gbr grc hun irl 6055 7632 5464 6993 8892 4925 836 5983 isr ita jpn kaz kor ltu mex nld 5538 4621 5278 6050 6667 5093 6306 5170 nor nzl per pol rus sgp svk svn 5128 6177 7289 9366 3892 5468 5723 5331 swe tur usa12_14 4469 5277 7921

$ncol aut bel can chl cze deu dnk ecu 1328 1328 1328 1328 1328 1328 1328 1328 esp est fin fra gbr grc hun irl 1328 1328 1328 1328 1328 1328 1328 1328 isr ita jpn kaz kor ltu mex nld 1328 1328 1328 1328 1328 1328 1328 1328 nor nzl per pol rus sgp svk svn 1328 1328 1328 1328 1328 1328 1328 1328 swe tur usa12_14 1328 1328 1328

piaac_a<-getData(piaac_all,varnames=c('lit','num','spfwt0','gender_r','ageg5lfs','cntryid'),addAttributes = T) There were 13 warnings (use warnings() to see them) warnings() Warning messages: 1: In getData(data = structure(list(userConditions = list(), ... : The requested dataset has 0 rows. 2: In getData(data = structure(list(userConditions = list(), ... : The requested dataset has 0 rows. 3: In getData(data = structure(list(userConditions = list(), ... : The requested dataset has 0 rows. 4: In getData(data = structure(list(userConditions = list(), ... : The requested dataset has 0 rows. 5: In getData(data = structure(list(userConditions = list(), ... : The requested dataset has 0 rows. 6: In getData(data = structure(list(userConditions = list(), ... : The requested dataset has 0 rows. 7: In getData(data = structure(list(userConditions = list(), ... : The requested dataset has 0 rows. 8: In getData(data = structure(list(userConditions = list(), ... : The requested dataset has 0 rows. 9: In getData(data = structure(list(userConditions = list(), ... : The requested dataset has 0 rows. 10: In getData(data = structure(list(userConditions = list(), ... : The requested dataset has 0 rows. 11: In getData(data = structure(list(userConditions = list(), ... : The requested dataset has 0 rows. 12: In getData(data = structure(list(userConditions = list(), ... : The requested dataset has 0 rows. 13: In getData(data = structure(list(userConditions = list(), ... : The requested dataset has 0 rows. dim(piaac_a) $nrow aut bel can chl cze deu dnk ecu 5025 4984 26683 0 6081 5379 7286 0 esp est fin fra gbr grc hun irl 5971 7586 5464 6907 8806 0 0 5963 isr ita jpn kaz kor ltu mex nld 0 4589 5173 0 6651 0 0 5082 nor nzl per pol rus sgp svk svn 4947 0 0 9366 3891 0 5702 0 swe tur usa12_14 4469 0 7759

$ncol aut bel can chl cze deu dnk ecu 104 104 104 104 104 104 104 104 esp est fin fra gbr grc hun irl 104 104 104 104 104 104 104 104 isr ita jpn kaz kor ltu mex nld 104 104 104 104 104 104 104 104 nor nzl per pol rus sgp svk svn 104 104 104 104 104 104 104 104 swe tur usa12_14 104 104 104

webervienna commented 2 years ago

Maybe that might be also of help. Here is the output trying the same for Singapore (just as an example from the round 2 or 3 countries). I have loaded PIAAC data from Singapore; So far all good. dim call gives a reasonable response. Creating a subset for some particular variables tells me the dataset has 0 rows - which is contradicting to the dim output and it is also not expected as I would expect a data.frame with about 1300 rows. I have tried also with listing the colnames to show you that the variables addressed here are included within the dataset.

R CODE AND OUTPUT for SINGAPORE

piaac_sgp<-readPIAAC('~/PIAAC/Cycle 1/',countries='sgp') Found cached data for country code “sgp”. dim(piaac_sgp) [1] 5468 1328 piaac_s<-getData(piaac_sgp,varnames=c('lit','num','spfwt0','gender_r','ageg5lfs','cntryid'),addAttributes = T) Warning message: In getData(piaac_sgp, varnames = c("lit", "num", "spfwt0", "gender_r", : The requested dataset has 0 rows. colnames(piaac_sgp) [1] "cntryid" "cntryid_e" "seqid" "age_r"
[5] "gender_r" "disp_cibq" "disp_main" "disp_mainwrc"
[9] "a_n01_t" "b_q01a" "b_q01a_t" "b_q01a3"
[13] "b_q01a3_c" "b_q01b" "b_q01c1" "b_q01c1_c"
[17] "b_q01c1_t" "b_q01c2" "b_q01d" "b_d01d"
[21] "b_d01d_c" "b_q02a" "b_q02a_t1" "b_q02a_t2"
[25] "b_q02b" "b_q02b_c" "b_q02c" "b_q03a"
[29] "b_q03b" "b_q03b_c" "b_q03c1" "b_q03c1_c"
[33] "b_q03c2" "b_q03d" "b_d03d" "b_d03d_c"
[37] "b_q04a" "b_q04b" "b_q04b_c" "b_q05a"
[41] "b_q05b" "b_q05c" "b_q05c_t" "b_q10a"
[45] "b_q10b" "b_q10c" "b_q11" "b_q12a"
[49] "b_q12a_t" "b_q12b" "b_q12c" "b_q12d"
[53] "b_q12d_c" "b_q12e" "b_q12f" "b_q12f_c"
[57] "b_q12g" "b_q12h" "b_q12h_c" "b_d12h"
[61] "b_q13" "b_q14a" "b_q14b" "b_q15a"
[65] "b_q15b" "b_q15c" "b_q16" "b_q17"
[69] "b_q18a" "b_q19a" "b_q20a" "b_q20b"
[73] "b_q26a" "b_q26a_t" "b_q26b" "c_q01a"
[77] "c_q01b" "c_q01c" "c_q02a" "c_q02b"
[81] "c_q02c" "c_q03_01" "c_q03_02" "c_q03_03"
[85] "c_q03_04" "c_q03_05" "c_q03_06" "c_q03_07"
[89] "c_q03_08" "c_q03_09" "c_q03_10" "c_s03"
[93] "c_q04a" "c_q04b" "c_q04c" "c_q04d"
[97] "c_q04e" "c_q04f" "c_q04g" "c_q04h"
[101] "c_q04i" "c_q04j" "c_d04" "c_q05"
[105] "c_d05" "c_q06" "c_d06" "c_q07"
[109] "c_q07_t" "c_q08a" "c_q08b" "c_q08c1"
[113] "c_q08c1_c" "c_q08c2" "c_d08c" "c_q09"
[117] "c_q09_c" "c_d09" "c_d09_t" "c_q10a"
[121] "c_q10a_c" "d_q03" "d_q04" "d_q04_t"
[125] "d_q04_t1" "d_q05a1" "d_q05a1_c" "d_q05a2"
[129] "d_q05a3" "d_q05b1" "d_q05b1_c" "d_q05b2"
[133] "d_q05b3" "d_q06a" "d_q06b" "d_q06c"
[137] "d_q07a" "d_q07b" "d_q07b_c" "d_q08a"
[141] "d_q08b" "d_q09" "d_q10" "d_q10_c"
[145] "d_q10_t" "d_q10_t1" "d_q11a" "d_q11b"
[149] "d_q11c" "d_q11d" "d_q12a" "d_q12b"
[153] "d_q12c" "d_q13a" "d_q13b" "d_q13c"
[157] "d_q14" "d_q16a" "d_s16a" "d_d16a"
[161] "d_q16b" "d_q16b_t" "d_q16c" "d_q16d1"
[165] "d_q16d2" "d_q16d3" "d_q16d4" "d_q16d5"
[169] "d_q16d6" "d_q17a" "d_q17b" "d_q17c"
[173] "d_q17d" "d_q18a" "d_q18a_t" "d_q18b"
[177] "d_q18c1" "d_q18c2" "e_q03" "e_q04"
[181] "e_q05a1" "e_q05a1_c" "e_q05a2" "e_q05b1"
[185] "e_q05b1_c" "e_q05b2" "e_q06" "e_q07a"
[189] "e_q07b" "e_q08" "e_q09" "e_q09_c"
[193] "e_q10" "f_q01b" "f_q02a" "f_q02b"
[197] "f_q02c" "f_q02d" "f_q02e" "f_q03a"
[201] "f_q03b" "f_q03c" "f_q04a" "f_q04b"
[205] "f_q05a" "f_q05b" "f_q06b" "f_q06c"
[209] "f_q07a" "f_q07b" "g_q01a" "g_q01a_t"
[213] "g_q01a_t1" "g_q01b" "g_q01b_t" "g_q01b_t1"
[217] "g_q01c" "g_q01c_t" "g_q01c_t1" "g_q01d"
[221] "g_q01e" "g_q01f" "g_q01f_t" "g_q01f_t1"
[225] "g_q01g" "g_q01g_t" "g_q01g_t1" "g_q01h"
[229] "g_q01h_t" "g_q01h_t1" "g_q02a" "g_q02b"
[233] "g_q02c" "g_q02d" "g_q03b" "g_q03c"
[237] "g_q03d" "g_q03f" "g_q03g" "g_q03h"
[241] "g_q04" "g_q04_t" "g_q05a" "g_q05c"
[245] "g_q05d" "g_q05e" "g_q05f" "g_q05g"
[249] "g_q05h" "g_q06" "g_q07" "g_q08"
[253] "h_q01a" "h_q01b" "h_q01b_t" "h_q01c"
[257] "h_q01c_t" "h_q01d" "h_q01e" "h_q01e_t"
[261] "h_q01f" "h_q01g" "h_q01h" "h_q02a"
[265] "h_q02b" "h_q02c" "h_q02d" "h_q03b"
[269] "h_q03c" "h_q03d" "h_q03f" "h_q03g"
[273] "h_q03h" "h_q04a" "h_q04b" "h_q05a"
[277] "h_q05c" "h_q05d" "h_q05e" "h_q05f"
[281] "h_q05g" "h_q05h" "i_q04b" "i_q04d"
[285] "i_q04h" "i_q04j" "i_q04l" "i_q04m"
[289] "i_q05f" "i_q06a" "i_q07a" "i_q07b"
[293] "i_q08" "i_q08_t" "j_q01" "j_q01_c"
[297] "j_q01_t" "j_q01_t1" "j_q02a" "j_q02c"
[301] "j_q03a" "j_q03b" "j_q03b_c" "j_q03c"
[305] "j_q03c_c" "j_q03d1" "j_q03d1_c" "j_q03d2"
[309] "j_q03d2_c" "j_q04a" "j_q04a_t" "j_q04c1"
[313] "j_q04c1_c" "j_q04c2" "j_q04c2_t" "j_q04c2_t1"
[317] "j_n05a2" "j_q06a" "j_q06a_t" "j_q06b"
[321] "j_q06b_t" "j_q07a" "j_q07a_t" "j_q07b"
[325] "j_q07b_t" "j_q08" "computerexperience" "nativespeaker"
[329] "edlevel3" "cba_core_stage1_score" "cba_core_stage2_score" "corestage1_pass"
[333] "corestage2_pass" "random_cba_module1" "random_cba_module2" "random_cba_module1_stage1" [337] "random_cba_module1_stage2" "random_cba_module2_stage1" "random_cba_module2_stage2" "cba_start"
[341] "ppc_score" "random_pp" "prc_pv_q1" "prc_sp_q1"
[345] "prc_pf_q1" "prc_pf_q2" "prc_pf_q3" "paper"
[349] "cbamod1" "cbamod2" "cbamod2alt" "cbamod1stg1"
[353] "cbamod2stg1" "cbamod1stg2" "cbamod2stg2" "monthlyincpr"
[357] "yearlyincpr" "pbroute" "zz1a" "zz1b_01"
[361] "zz1b_02" "zz2" "zz3" "zz4_01"
[365] "zz4_02" "zz4_03" "zz4_04" "zz4_05"
[369] "zz4_06" "zz5" "zz6" "isced_hf"
[373] "isced_hf_c" "isco08_c" "isco08_l" "isic4_c"
[377] "isic4_l" "lng_l1" "lng_l2" "lng_home"
[381] "cnt_h" "cnt_brth" "reg_tl2" "lng_bq"
[385] "lng_ci" "yrsqual" "yrsqual_t" "yrsget"
[389] "vet" "ctryqual" "birthrgn" "firlgrgn"
[393] "seclgrgn" "homlgrgn" "forbornlang" "pared"
[397] "nativelang" "bornlang" "natbilang" "forbilang"
[401] "homlang" "ctryrgn" "impar" "imgen"
[405] "imyrs" "imyrs_c" "imyrcat" "ageg5lfs"
[409] "ageg10lfs" "ageg10lfs_t" "edcat8" "edcat7"
[413] "edcat6" "leaver1624" "leavedu" "fe12"
[417] "aetpop" "faet12" "faet12jr" "faet12njr"
[421] "nfe12" "nfe12jr" "nfe12njr" "fnfaet12"
[425] "fnfe12jr" "fnfaet12jr" "fnfaet12njr" "edwork"
[429] "neet" "nfehrsnjr" "nfehrsjr" "nfehrs"
[433] "nopaidworkever" "paidwork12" "paidwork5" "iscoskil4"
[437] "isic1l" "isic2l" "isic1c" "isic2c"
[441] "isco1c" "isco2c" "isco1l" "isco2l"
[445] "earnhr" "earnhrdcl" "earnhrppp" "earnhrbonus"
[449] "earnhrbonusdcl" "earnhrbonusppp" "earnmth" "earnmthppp"
[453] "earnmthselfppp" "earnmthbonus" "earnmthall" "earnmthalldcl"
[457] "earnmthallppp" "earnmthbonusppp" "earnflag" "learnatwork"
[461] "learnatwork_wle_ca" "readytolearn" "readytolearn_wle_ca" "icthome"
[465] "icthome_wle_ca" "ictwork" "ictwork_wle_ca" "influence"
[469] "influence_wle_ca" "numhome" "numhome_wle_ca" "numwork"
[473] "numwork_wle_ca" "planning" "planning_wle_ca" "readhome"
[477] "readhome_wle_ca" "readwork" "readwork_wle_ca" "taskdisc"
[481] "taskdisc_wle_ca" "writhome" "writhome_wle_ca" "writwork"
[485] "writwork_wle_ca" "c301c05s" "c301c05t" "c301c05f"
[489] "c301c05a" "c300c02s" "c300c02t" "c300c02f"
[493] "c300c02a" "d302c02s" "d302c02t" "d302c02f"
[497] "d302c02a" "c600c04s" "c600c04t" "c600c04f"
[501] "c600c04a" "c601c06s" "c601c06t" "c601c06f"
[505] "c601c06a" "e645001s" "e645001t" "e645001f"
[509] "e645001a" "d311701s" "d311701t" "d311701f"
[513] "d311701a" "c308120s" "c308120t" "c308120f"
[517] "c308120a" "e321001s" "e321001t" "e321001f"
[521] "e321001a" "e321002s" "e321002t" "e321002f"
[525] "e321002a" "c305215s" "c305215t" "c305215f"
[529] "c305215a" "c305218s" "c305218t" "c305218f"
[533] "c305218a" "c308117s" "c308117t" "c308117f"
[537] "c308117a" "c308119s" "c308119t" "c308119f"
[541] "c308119a" "c308121s" "c308121t" "c308121f"
[545] "c308121a" "c308118s" "c308118t" "c308118f"
[549] "c308118a" "d304710s" "d304710t" "d304710f"
[553] "d304710a" "d304711s" "d304711t" "d304711f"
[557] "d304711a" "d315512s" "d315512t" "d315512f"
[561] "d315512a" "e327001s" "e327001t" "e327001f"
[565] "e327001a" "e327002s" "e327002t" "e327002f"
[569] "e327002a" "e327003s" "e327003t" "e327003f"
[573] "e327003a" "e327004s" "e327004t" "e327004f"
[577] "e327004a" "c308116s" "c308116t" "c308116f"
[581] "c308116a" "c309320s" "c309320t" "c309320f"
[585] "c309320a" "c309321s" "c309321t" "c309321f"
[589] "c309321a" "d307401s" "d307401t" "d307401f"
[593] "d307401a" "d307402s" "d307402t" "d307402f"
[597] "d307402a" "c313412s" "c313412t" "c313412f"
[601] "c313412a" "c313414s" "c313414t" "c313414f"
[605] "c313414a" "c309319s" "c309319t" "c309319f"
[609] "c309319a" "c309322s" "c309322t" "c309322f"
[613] "c309322a" "e322001s" "e322001t" "e322001f"
[617] "e322001a" "e322002s" "e322002t" "e322002f"
[621] "e322002a" "e322005s" "e322005t" "e322005f"
[625] "e322005a" "e320001s" "e320001t" "e320001f"
[629] "e320001a" "e320003s" "e320003t" "e320003f"
[633] "e320003a" "e320004s" "e320004t" "e320004f"
[637] "e320004a" "c310406s" "c310406t" "c310406f"
[641] "c310406a" "c310407s" "c310407t" "c310407f"
[645] "c310407a" "e322003s" "e322003t" "e322003f"
[649] "e322003a" "e323003s" "e323003t" "e323003f"
[653] "e323003a" "e323004s" "e323004t" "e323004f"
[657] "e323004a" "e322004s" "e322004t" "e322004f"
[661] "e322004a" "d306110s" "d306110t" "d306110f"
[665] "d306110a" "d306111s" "d306111t" "d306111f"
[669] "d306111a" "c313410s" "c313410t" "c313410f"
[673] "c313410a" "c313411s" "c313411t" "c313411f"
[677] "c313411a" "c313413s" "c313413t" "c313413f"
[681] "c313413a" "e318001s" "e318001t" "e318001f"
[685] "e318001a" "e318003s" "e318003t" "e318003f"
[689] "e318003a" "e323002s" "e323002t" "e323002f"
[693] "e323002a" "e323005s" "e323005t" "e323005f"
[697] "e323005a" "e329002s" "e329002t" "e329002f"
[701] "e329002a" "e329003s" "e329003t" "e329003f"
[705] "e329003a" "c615602s" "c615602t" "c615602f"
[709] "c615602a" "c615603s" "c615603t" "c615603f"
[713] "c615603a" "c624619s" "c624619t" "c624619f"
[717] "c624619a" "c624620s" "c624620t" "c624620f"
[721] "c624620a" "c604505s" "c604505t" "c604505f"
[725] "c604505a" "c605506s" "c605506t" "c605506f"
[729] "c605506a" "c605507s" "c605507t" "c605507f"
[733] "c605507a" "c605508s" "c605508t" "c605508f"
[737] "c605508a" "e650001s" "e650001t" "e650001f"
[741] "e650001a" "c623616s" "c623616t" "c623616f"
[745] "c623616a" "c623617s" "c623617t" "c623617f"
[749] "c623617a" "c619609s" "c619609t" "c619609f"
[753] "c619609a" "e657001s" "e657001t" "e657001f"
[757] "e657001a" "e646002s" "e646002t" "e646002f"
[761] "e646002a" "c620610s" "c620610t" "c620610f"
[765] "c620610a" "c620612s" "c620612t" "c620612f"
[769] "c620612a" "e632001s" "e632001t" "e632001f"
[773] "e632001a" "e632002s" "e632002t" "e632002f"
[777] "e632002a" "c607510s" "c607510t" "c607510f"
[781] "c607510a" "c614601s" "c614601t" "c614601f"
[785] "c614601a" "c618607s" "c618607t" "c618607f"
[789] "c618607a" "c618608s" "c618608t" "c618608f"
[793] "c618608a" "e635001s" "e635001t" "e635001f"
[797] "e635001a" "c613520s" "c613520t" "c613520f"
[801] "c613520a" "c608513s" "c608513t" "c608513f"
[805] "c608513a" "e655001s" "e655001t" "e655001f"
[809] "e655001a" "c602501s" "c602501t" "c602501f"
[813] "c602501a" "c602502s" "c602502t" "c602502f"
[817] "c602502a" "c602503s" "c602503t" "c602503f"
[821] "c602503a" "c611516s" "c611516t" "c611516f"
[825] "c611516a" "c611517s" "c611517t" "c611517f"
[829] "c611517a" "c606509s" "c606509t" "c606509f"
[833] "c606509a" "e665001s" "e665001t" "e665001f"
[837] "e665001a" "e665002s" "e665002t" "e665002f"
[841] "e665002a" "c622615s" "c622615t" "c622615f"
[845] "c622615a" "e636001s" "e636001t" "e636001f"
[849] "e636001a" "c617605s" "c617605t" "c617605f"
[853] "c617605a" "c617606s" "c617606t" "c617606f"
[857] "c617606a" "e641001s" "e641001t" "e641001f"
[861] "e641001a" "e661001s" "e661001t" "e661001f"
[865] "e661001a" "e661002s" "e661002t" "e661002f"
[869] "e661002a" "e660003s" "e660003t" "e660003f"
[873] "e660003a" "e660004s" "e660004t" "e660004f"
[877] "e660004a" "e634001s" "e634001t" "e634001f"
[881] "e634001a" "e634002s" "e634002t" "e634002f"
[885] "e634002a" "e651002s" "e651002t" "e651002f"
[889] "e651002a" "e664001s" "e664001t" "e664001f"
[893] "e664001a" "e644002s" "e644002t" "e644002f"
[897] "e644002a" "c612518s" "c612518t" "c612518f"
[901] "c612518a" "u01a000a" "u01a000f" "u01a000t"
[905] "u01b000a" "u01b000f" "u01b000t" "u03a000a"
[909] "u03a000f" "u03a000t" "u06a000a" "u06a000f"
[913] "u06a000t" "u06b000a" "u06b000f" "u06b000t"
[917] "u21x000a" "u21x000f" "u21x000t" "u04a000a"
[921] "u04a000f" "u04a000t" "u19a000a" "u19a000f"
[925] "u19a000t" "u19b000a" "u19b000f" "u19b000t"
[929] "u07x000a" "u07x000f" "u07x000t" "u02x000a"
[933] "u02x000f" "u02x000t" "u16x000a" "u16x000f"
[937] "u16x000t" "u11b000a" "u11b000f" "u11b000t"
[941] "u23x000a" "u23x000f" "u23x000t" "u01a000s"
[945] "u01b000s" "u02x000s" "u03a000s" "u04a000s"
[949] "u06a000s" "u06b000s" "u07x000s" "u11b000s"
[953] "u16x000s" "u19a000s" "u19b000s" "u21x000s"
[957] "u23x000s" "m301c05s" "p330001s" "n302c02s"
[961] "m600c04s" "m300c02s" "p601c06s" "p614601s"
[965] "p645001s" "n306110s" "n306111s" "m313410s"
[969] "m313411s" "m313412s" "m313413s" "m313414s"
[973] "p324002s" "p324003s" "m305215s" "m305218s"
[977] "p317001s" "p317002s" "p317003s" "m310406s"
[981] "m310407s" "m309319s" "m309320s" "m309321s"
[985] "m309322s" "m615602s" "m615603s" "p640001s"
[989] "m620610s" "m620612s" "p666001s" "m623616s"
[993] "m623617s" "m623618s" "m624619s" "m624620s"
[997] "m618607s" "m618608s" "m604505s" "m610515s"
[ reached getOption("max.print") -- omitted 328 entries ]

pdbailey0 commented 2 years ago

@webervienna, it seems that the file international-codebook.xlsx in the values tab does not have a level for Singapore for the variable cntryid (which is 702).

image

While new data may eventually be released, I'll offer a simple alternative: you could also go to the "Review" tab, unprotect that sheet, and add Singapore to your codebook.

image

I tried this and then my file looked like this:

image

you then need to force a reread of the data

piaac_sgp<-readPIAAC('~/PIAAC/Cycle 1/',countries='sgp', forceReread=T)

let me know if that solves your problem.

webervienna commented 2 years ago

Thank you very much that was a brilliant idea. I have added all countries and their code from round 2 and 3 which are actually missing. Now it works fine. Thank you sooo much!

pdbailey0 commented 2 years ago

@webervienna we're going to release a new EdSurvey version that fixes this so that codes not in the code book appear as the codes themselves (so you'd see 702 if you did not update the code book).

We're adding this to the EdSurvey News. Is this an okay to credit you? I'd happily update it to your name.

 \item \code{readPIAAC} is now set to validate variables with missing value labels undefined in the codebook.  For missing label values it will now return the numeric value as the label instead of an NA value.  Thanks to GitHub user webervienna for identifying and reporting this bug.
webervienna commented 2 years ago

Oh that's very nice thank you!