search

mhahsler / pomdp

R package for Partially Observable Markov Decision Processes
16 stars 5 forks source link

Importing transition probabilities from excel #9

Closed haniees closed 3 years ago

haniees commented 3 years ago

Hello,

I am trying to use an imported excel file for my transition and observation probabilities, There are 1616 transition probability, and 4 16 observation probabilities that I imported. Here is the code: Total <- POMDP( name = "Total", discount = 0.99, horizon = c(first = 1, second = 1, third =1, fourth = 1), states = c("s1", "s2", "s3", "s4", "s5","s6","s7", "s8", "s9","s10","s11","s12","s13","s14","s15","s16"), actions = c("Physio", "Gait", "cardiovascular", "nothing"), observations = c("o1", "o2", "o3", "o4"), start = "uniform", transition_prob = list( first = list( "Physio" = as.matrix(PF_THA_Trans_0), "Gait" = as.matrix(PF_THA_Trans_0), "cardiovascular" = as.matrix(PF_THA_Trans_0), "nothing" = "uniform"), second = list( "Physio" = as.matrix(PF_THA_Trans_1), "Gait" = as.matrix(PF_THA_Trans_1), "cardiovascular" = as.matrix(PF_THA_Trans_1), "nothing" = "uniform"), third = list( "Physio" = as.matrix(PF_THA_Trans_2), "Gait" = as.matrix(PF_THA_Trans_2), "cardiovascular" = as.matrix(PF_THA_Trans_2), "nothing" = "uniform"), fourth = list( "Physio" = as.matrix(PF_THA_Trans_3), "Gait" = as.matrix(PF_THA_Trans_3), "cardiovascular" = as.matrix(PF_THA_Trans_3), "nothing" = "uniform")),

observation_prob = list( first = list( "Physio" = as.matrix(PF_THA_obs_1), "Gait" = as.matrix(PF_THA_obs_1), "cardiovascular" = as.matrix(PF_THA_obs_1), "nothing" = "uniform"), second = list( "Physio" = as.matrix(PF_THA_obs_2), "Gait" = as.matrix(PF_THA_obs_2), "cardiovascular" = as.matrix(PF_THA_obs_2), "nothing" = "uniform"), third = list( "Physio" = as.matrix(PF_THA_obs_3), "Gait" = as.matrix(PF_THA_obs_3), "cardiovascular" = as.matrix(PF_THA_obs_3), "nothing" = "uniform"), fourth = list( "Physio" = as.matrix(PF_THA_obs_4), "Gait" = as.matrix(PF_THA_obs_4), "cardiovascular" = as.matrix(PF_THA_obs_4), "nothing" = "uniform")),

the reward helper expects: action, start.state, end.state, observation, value

reward = rbind( R("Physio", "s1", v = 22.5), R("Physio", "s2", v = 67.5), R("Physio", "s3", v = 112.5), R("Physio", "s4", v = 157.5), R("Physio", "s5", v = 202.5), R("Physio", "s6", v = 247.5), R("Physio", "s7", v = 337.5), R("Physio", "s8", v = 382.5), R("Physio", "s9", v = 427.5), R("Physio", "s10", v = 472.5), R("Physio", "s11", v = 517.5), R("Physio", "s12", v = 562.5), R("Physio", "s13", v = 607.5), R("Physio", "s14", v = 652.5), R("Physio", "s15", v = 697.5), R("Physio", "s16", v = 742.5), R("Gait", "s1", v = 22.5), R("Gait", "s2", v = 67.5), R("Gait", "s3", v = 112.5), R("Gait", "s4", v = 157.5), R("Gait", "s5", v = 202.5), R("Gait", "s6", v = 247.5), R("Gait", "s7", v = 337.5), R("Gait", "s8", v = 382.5), R("Gait", "s9", v = 427.5), R("Gait", "s10", v = 472.5), R("Gait", "s11", v = 517.5), R("Gait", "s12", v = 562.5), R("Gait", "s13", v = 607.5), R("Gait", "s14", v = 652.5), R("Gait", "s15", v = 697.5), R("Gait", "s16", v = 742.5), R("cardiovascular", "s1", v = 22.5), R("cardiovascular", "s2", v = 67.5), R("cardiovascular", "s3", v = 112.5), R("cardiovascular", "s4", v = 157.5), R("cardiovascular", "s5", v = 202.5), R("cardiovascular", "s6", v = 247.5), R("cardiovascular", "s7", v = 337.5), R("cardiovascular", "s8", v = 382.5), R("cardiovascular", "s9", v = 427.5), R("cardiovascular", "s10", v = 472.5), R("cardiovascular", "s11", v = 517.5), R("cardiovascular", "s12", v = 562.5), R("cardiovascular", "s13", v = 607.5), R("cardiovascular", "s14", v = 652.5), R("cardiovascular", "s15", v = 697.5), R("cardiovascular", "s16", v = 742.5), R("nothing", "s1", v = 22.5), R("nothing", "s2", v = 67.5), R("nothing", "s3", v = 112.5), R("nothing", "s4", v = 157.5), R("nothing", "s5", v = 202.5), R("nothing", "s6", v = 247.5), R("nothing", "s7", v = 337.5), R("nothing", "s8", v = 382.5), R("nothing", "s9", v = 427.5), R("nothing", "s10", v = 472.5), R("nothing", "s11", v = 517.5), R("nothing", "s12", v = 562.5), R("nothing", "s13", v = 607.5), R("nothing", "s14", v = 652.5), R("nothing", "s15", v = 697.5), R("nothing", "s16", v = 742.5)
), max = TRUE, ) Total sol <- solve_POMDP(model = Total, discount = 0.99, method = "enum") sol policy(sol)

I got this error which I do not know what does it mean: Error in NextMethod("[") : object 'i' not found

I know that all the data are the same for different actions for now but I don't think that is the problem. Could you please help me with this? Thanks in advance

mhahsler commented 3 years ago

Hi,

the latest version on GitHub runs with your code. Enum might take a while.