missing beliefs stops plot_policy_graph

[meeheal]

Hi Michael,

This is likely not a bug it may have been an oversight or a design choice. After updating to pomdp 0.99.3 the function plot_policy_graph stopped plotting whenever one of the nodes had no belief attached to it. It used to give a warning, such as, missing belief points for node(s): 1, 3, 4,..., Increase parameter fg_points, but it would still plot the resulting graph using a uniform distribution for the belief associated to each of those nodes. The updated version stops instead of giving a warning.

I know I can still produce the graphs by including beliefs = FALSE argument into the function, but that removes all of the beliefs for every node, including those for which there was a well-defined belief.

Is there a specific reason for this design choice? Any way of having a mixed beliefs argument option so plot_policy_graph would still plot the graphs using the old method of assigning a uniform distribution to nodes that did not have a well-defined belief?

Thanks for the consideration!

[mhahsler]

This code was changed to make sure that the graph is only created if the POMDP policy converged. There are some issues and one are missing belief points. The error message should still read something like this: "No belief points for policy graph node(s): xxx. Increase the number for parameter belief." Maybe we could make it a warning again and grey the missing belief points out.

[meeheal]

Hi Michael,

I can describe what I have done. Yes that is the error message I receive and I think I have tried everything to get belief points for the nodes in the graph that are missing belief points, namely upping the argument "parameter = list(fg_points = xxx)".

I thought that depending on the method, grid and sarsop are what I have used, the stopping of the process could mean that it there is convergence.

For instance, I have Tag POMDP running using grid and with fg_points = 200 it converged, i.e., process stops, in about 4000 s (discount = 0.9). Whereas with sarsop, since you can specify a timeout parameter, for example 1000 s, then despite the stopping of the process, converged = FALSE. Since while running the grid method the total expected reward is not as large as with sarsop, I have also tried it with fg_points = 250, and left it running for a whole weekend with nothing. I am forced to stop the process manually. However, the issue I'm getting is the approximately optimal policy with fg_points = 200, method = 'grid' (aka the default method) does not go through the plot_policy_graph function unless I include the belief = FALSE argument.

[emile-pelletier-gc]

Hi Michael,

I am running into a problem trying to get the policy graph plotted with beliefs. I seem to always get the result plotted if I use plot_policy_graph(sol, belief = FALSE) but I am hoping to have the underlying estimated belief centre (representing the node in the graph) showing up as either the colored pie (using engine igraph) or as a mouse-over table with colored nodes (using engine visNetwork).

If I try plot_policy_graph(sol) I get the error:

Error in policy_graph(x, belief = belief, col = col) : No belief points for policy graph node(s): 1, 2. Increase the number for parameter belief.

On this error, I have understood what you kindly pointed out: this issue needs an increase for the belief parameter (I was originally thinking you meant more fg_points in solve_POMDP). So now I run this: plot_policy_graph(sol, belief = 10000) and I get this error

Error in Ob[[action]][, observation, drop = FALSE] : incorrect number of dimensions

Here is a reproduceable example

ex_POMDP.txt First resave as a ".r" source file. In R Studio (which is what I am using) I would source this file and then run this first to get a solved POMDP

sol <- solve_POMDP(ex_POMDP) 

I also wanted to add that this package is really fantastic and is helping me get much more knowledgeable about POMDPs. Thank you!