transition_matrix permutated

[meeheal]

Hi Farzad,

I put my POMDP object as an argument in the function (transition_matrix(objPOMDP)$myAction) and the resulting matrix has rows and columns permutated. I have four states and three actions. It appears the states are being reordered to [2, 3, 4, 1] which incidentally then matches an alphabetical ordering of the four states, but not the state factor ordering.

This is a real problem because the row and column names of transition_matrix(objPOMDP)$myAction are not reordered [2, 3, 4, 1], so the result is wrong.

I am using Heatmap to plot it, and for it to be correct I need the following code: heatmap(transition_matrix(objPOMDP)$myAction)[c(2, 3, 4, 1), c(2, 3, 4, 1)], labRow = objPOMDP$model$states, labCol = objPOMDP$model$states, revC = TRUE, Rowv = NA, Colv = NA).

Everything is correct when I use the following: objPOMDP$model$transition_prob[objPOMDP$model$transition_prob$action == "myAction",]

This issue does not arise when using observation_matrix; here, the result maintains the state factor ordering.

Any ideas how this could be happening? Is this a bug?

Thanks, Emile

[mhahsler]

Hi Emile! Thank you for the bug report. Please post a minimal example that shows the problem. This will speed up the process of fixing the bug.

Regards, Michael

[meeheal]

I am attaching a text file that is a script file for [r] that I believe will run.

I am realizing a bit more about what I think is happening, in that the order in which the lines are entered matters to the transitionmatrix support function, so it appears as if it is reading it and interpreting it as if the order of the T(...) lines corresponds with the state pairs in the following permutation (1, 1), (1, 2), (1, 3), (2, 1), ..., (3, 3).

It is also happening with the observation_matrix support function, contrary to what I originally posted.

Thanks, Emile

ExampleBug - Copy.txt

[mhahsler]

I get the following when I run your code and it seems correct to me. The order in which you specify the T calls should not matter.

> transition_matrix(objPOMDP)$Start
               A-first-state A-plus-1-state A-plus-2-state
A-first-state           0.40            0.5           0.10
A-plus-1-state          0.15            0.8           0.05
A-plus-2-state          0.15            0.3           0.55
> objPOMDP$model$transition_prob[objPOMDP$model$transition_prob$action == "Start",]
  action    start.state      end.state probability
1  Start A-plus-1-state A-plus-1-state        0.80
2  Start A-plus-1-state A-plus-2-state        0.05
3  Start A-plus-1-state  A-first-state        0.15
4  Start  A-first-state A-plus-1-state        0.50
5  Start  A-first-state A-plus-2-state        0.10
6  Start  A-first-state  A-first-state        0.40
7  Start A-plus-2-state A-plus-1-state        0.30
8  Start A-plus-2-state A-plus-2-state        0.55
9  Start A-plus-2-state  A-first-state        0.15

[meeheal]

Interesting Michael, that's not what I've been getting, which is:

 transition_matrix(objPOMDP)$Start
               A-first-state A-plus-1-state A-plus-2-state
A-first-state            0.8           0.05           0.15
A-plus-1-state           0.5           0.10           0.40
A-plus-2-state           0.3           0.55           0.15
> #with this
> objPOMDP$model$transition_prob[objPOMDP$model$transition_prob$action == "Start",]
  action    start.state      end.state probability
1  Start A-plus-1-state A-plus-1-state        0.80
2  Start A-plus-1-state A-plus-2-state        0.05
3  Start A-plus-1-state  A-first-state        0.15
4  Start  A-first-state A-plus-1-state        0.50
5  Start  A-first-state A-plus-2-state        0.10
6  Start  A-first-state  A-first-state        0.40
7  Start A-plus-2-state A-plus-1-state        0.30
8  Start A-plus-2-state A-plus-2-state        0.55
9  Start A-plus-2-state  A-first-state        0.15

1) Do I have an older POMDP package; 2) Is the combination of packages I have loaded a possible issue; 3) Is having used my main, larger code, inside Shiny FlexDashboard somehow the reason;

I'm going to try restarting R and R Studio, and if the problem is resolved, I will try to see if there's any way I can get this error again, and, if you are interested, I can let you know what I find.

I wouldn't be troubling anyone with this if the result you got is what I had been getting; that's exactly what I was trying to obtain.

[meeheal]

Restarting R and Updating Packages did nothing to fix the issue for me.

On my install at least, I think the factor order of model$states is changing depending on where it is being used (internally). So for the numerical entries in the transition_matrix, $states is (c("A-plus-1-state", "A-first-state", A-plus-2-state"), but for the row.names and col.names it uses the original, desired ordering: c("A-first-state", "A-plus-1-state", "A-plus-2-state").

My new code (to avoid possibly getting wrong vis, as recall I am then putting into a heatmap) is as follows:

heatmap(acast(melt(objPOMDP$model$transition_prob[objPOMDP$model$transition_prob$action == "Start",2:4]), start.state ~ end.state)[c("A-first-state", "A-plus-1-state", "A-plus-2-state"), c("A-first-state", "A-plus-1-state", "A-plus-2-state")], Rowv = NA, Colv = NA, revC = TRUE, col = colorRampPalette(brewer.pal(8, "Blues"))(36), margins = c(14, 15), scale = "none")

instead of
heatmap(transition_matrix(objPOMDP)$Start, Rowv = NA, Colv = NA, revC = TRUE, col = colorRampPalette(brewer.pal(8, "Blues"))(36), margins = c(14, 15), scale = "none") <Requires {reshape2, RColorBrewer}>

[mhahsler]

Please send me the output of sessionInfo() after pomdp is loaded.

[meeheal]

sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 17763)

Matrix products: default

locale:
[1] LC_COLLATE=English_Canada.1252  LC_CTYPE=English_Canada.1252   
[3] LC_MONETARY=English_Canada.1252 LC_NUMERIC=C                   
[5] LC_TIME=English_Canada.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] pomdp_0.99.0       RColorBrewer_1.1-2 reshape2_1.4.4     forcats_0.5.0     
 [5] stringr_1.4.0      purrr_0.3.4        readr_1.3.1        tidyr_1.1.0       
 [9] tibble_3.0.1       ggplot2_3.3.0      tidyverse_1.3.0    dplyr_0.8.5       

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.4.6          lubridate_1.7.8       lattice_0.20-38       assertthat_0.2.1     
 [5] digest_0.6.25         utf8_1.1.4            R6_2.4.1              cellranger_1.1.0     
 [9] plyr_1.8.6            backports_1.1.7       reprex_0.3.0          evaluate_0.14        
[13] httr_1.4.1            pillar_1.4.4          rlang_0.4.6           curl_4.3             
[17] readxl_1.3.1          rstudioapi_0.11       blob_1.2.1            rmarkdown_2.1        
[21] flexdashboard_0.5.1.1 igraph_1.2.5          munsell_0.5.0         Ternary_1.1.4        
[25] broom_0.5.6           compiler_3.6.3        modelr_0.1.8          xfun_0.14            
[29] askpass_1.1           pkgconfig_2.0.3       htmltools_0.4.0       openssl_1.4.1        
[33] tidyselect_1.1.0      fansi_0.4.1           viridisLite_0.3.0     crayon_1.3.4         
[37] dbplyr_1.4.4          withr_2.2.0           grid_3.6.3            nlme_3.1-144         
[41] jsonlite_1.6.1        gtable_0.3.0          lifecycle_0.2.0       DBI_1.1.0            
[45] magrittr_1.5          scales_1.1.1          cli_2.0.2             stringi_1.4.6        
[49] fs_1.4.1              xml2_1.3.2            ellipsis_0.3.1        generics_0.0.2       
[53] vctrs_0.3.0           tools_3.6.3           glue_1.4.1            hms_0.5.3            
[57] rsconnect_0.8.16      yaml_2.2.1            colorspace_1.4-1      rvest_0.3.5          
[61] knitr_1.28            haven_2.3.0         

[mhahsler]

Hmm, I use R4.0 and that might make a difference. I will check that and get back to you.

[mhahsler]

I can replicate the problem on R 3.6.3 and will work on a fix.

[mhahsler]

I have fixed the problem in the development version on GitHub. Here is the binary package for Windows. Please install it and check that it fixes the problem on your system.

pomdp_0.99.0-1.zip

[meeheal]

Hi Michael,

I did install the binary zip package and ran the same script as before, and, alas now it is returning what it should be! Thanks!

Emile

[mhahsler]

Great! Thanks for checking.