Memory limits

[awiktor]

SARSOP runs fine when I test it on a reduced problem with 50 states, 1681 observations, and 4 actions. However, every time I try to scale up the problem I get a memory error like the one below, always around 141-144 seconds. The full problem has 800 states, 6561 observations, and 4 actions, which I don't think should be too much for SARSOP to handle. I've tried changing the "memory" option, but when I set it to a very large number (eg 25000, my computer has 32GB of ram) the solver ends even earlier. Any idea what's going on?

############################################

Generating a pomdpx file: model.pomdpx

Loading the model ... input file : model.pomdpx loading time : 15.01s

SARSOP initializing ... initialization time : 0.52s

---

Time |#Trial |#Backup |LBound |UBound |Precision |#Alphas |#Beliefs

0.52 0 0 -27.2453 19.2808 46.5261 4 1
5.34 3 57 -7.50296 18.5921 26.095 61 30
11.436 6 100 -0.4758 18.592 19.0679 102 54
18.698 10 150 4.35311 18.591 14.2379 152 76
27.663 13 200 6.52428 18.5908 12.0665 201 101
39.416 16 250 7.60615 18.5897 10.9836 251 126
54.1 20 303 8.55015 18.5895 10.0393 304 151
72.41 25 350 9.17944 18.5893 9.40984 351 176
91.819 28 403 9.87546 18.5892 8.71378 404 200
112.769 31 451 10.2944 18.589 8.2946 452 224
141.689 35 500 10.8889 18.5889 7.69992 501 250
Memory limit reached, trying to write out policy

---

Time |#Trial |#Backup |LBound |UBound |Precision |#Alphas |#Beliefs

165.799 38 550 11.3859 18.5888 7.20282 551 273

Writing out policy ... output file : out.policy

[mykelk]

Wow, that's a lot of observations. Are some of the state variables fully observable? If so, can you frame it as a MOMDP?

[zsunberg]

Looks like that error is thrown here: https://github.com/AdaCompNUS/sarsop/blob/3ad76c45f949191e0ced3810b33da96a43f0b0da/src/Algorithms/SARSOP/SARSOP.cpp#L405

It might be worth asking the authors of the original C++ code about it.

Regarding the memory option, I don't know what the default behavior is, but it may default to using all of the computer's ram, in which case ~25 GB would be less than 32 GB explaining the early termination. I'm not sure if the C++ implementation has a debug mode (if it does, it appears that we did not give access to it through julia), but that might give more info.

[zsunberg]

@mykelk is probably right that reformulating the problem is the most likely way to get it to work

[awiktor]

I tried shrinking the problem to 800 states, 289 observations, and 4 actions, and I'm still getting the memory error. Based on the Sarsop paper, I feel like it should be able to handle substantially larger spaces than that. Also, when I monitor the ram usage it never goes above 15% of available memory. Could there be something else going on?

[mykelk]

Yes, agreed. Can you send us your pomdpx file? Then we can try running it and seeing where it gets stuck.

[awiktor]

It's 100MB. What's the best way to get it to you?

[mykelk]

Google drive or Box.

[zsunberg]

You also may be able to compress it significantly.

On Tue, Mar 13, 2018 at 10:06 AM Mykel Kochenderfer < notifications@github.com> wrote:

Google drive or Box.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/JuliaPOMDP/SARSOP.jl/issues/20#issuecomment-372742466, or mute the thread https://github.com/notifications/unsubscribe-auth/AEC0a9cslHPMgOUNH3yhGKqMXkt1I9-2ks5td_yUgaJpZM4SoB3z .

[awiktor]

https://stanford.box.com/s/7h3zmxozewdrj85aug0n4hvh1f1khuma

Thanks!

[mykelk]

What is the output that you get on that file? I tried running pomdpsol model.pomdpx and it seems to work just fine (at least as long as I've been waiting):

$ ./pomdpsol model.pomdpx

Loading the model ...
  input file   : model.pomdpx
  loading time : 13.34s

SARSOP initializing ...
  initialization time : 0.05s

-------------------------------------------------------------------------------
 Time   |#Trial |#Backup |LBound    |UBound    |Precision  |#Alphas |#Beliefs
-------------------------------------------------------------------------------
 0.047   0       0        -5.88503   12.8221    18.7072     4        1
 1.484   1       57       -3.9808    10.3291    14.3099     56       30
 4.063   2       117      -3.58901   10.204     13.793      108      60
 8.313   3       175      -3.33741   10.172     13.5094     166      89
 11.672  3       200      -3.33741   10.172     13.5094     190      114
 16.594  4       250      -3.28891   10.1128    13.4017     234      134
 22.875  5       300      -3.22491   10.101     13.3259     280      158
 30      6       350      -3.14038   10.092     13.2324     328      182
 36.86   7       400      -3.11357   10.082     13.1956     373      201
 47.266  8       457      -3.00448   10.0803    13.0848     425      229
 58.672  9       511      -2.95437   10.0688    13.0232     479      256
 69.015  10      555      -2.88664   10.0581    12.9448     514      277
 83.782  11      611      -2.88009   10.0568    12.9369     563      305
 102.344 12      673      -2.86133   10.0194    12.8807     612      335
 114.594 12      700      -2.86133   10.0194    12.8807     636      362
 131.422 13      750      -2.85017   10.0187    12.8689     679      383
 149.469 14      800      -2.6557    10.0174    12.6731     724      407
 171.125 15      850      -2.62163   10.0174    12.639      763      430
 190.594 16      900      -2.60584   10.0173    12.6232     804      449
 212.406 17      950      -2.56364   10.0173    12.5809     844      473
 234.797 18      1000     -2.52859   10.0172    12.5458     881      494
 256.688 19      1051     -2.52095   10.0172    12.5381     932      517

[awiktor]

Running the same command, mine stops after:

221.646 19 1051 -2.52095 10.0172 12.5381 922 517 Memory limit reached, trying to write out policy

[mykelk]

I terminated it after a while and got this:

 3260.31 60      3303     -2.35084   9.98152    12.3324     2612     1408
 3367.16 61      3359     -2.35084   9.98152    12.3324     2668     1426
 3477.78 62      3415     -2.35084   9.98152    12.3324     2724     1444
 3621.95 63      3463     -2.35084   9.98152    12.3324     2703     1459
*** Received SIGINT. User pressed control-C. ***

Terminating ...

-------------------------------------------------------------------------------
 Time   |#Trial |#Backup |LBound    |UBound    |Precision  |#Alphas |#Beliefs
-------------------------------------------------------------------------------
 3655.39 63      3478     -2.35084   9.98152    12.3324     2709     1465
-------------------------------------------------------------------------------

Writing out policy ...
  output file : out.policy

I compiled appl in cygwin. You can download the executable and the resulting policy here

You should be able to just drop in that pomdpsol file. You might need to install cygwin. Let me know how it goes!

[awiktor]

Looks like its working now! I had tried windows command prompt, Jupyter, and Julia REPL before, but I guess cygwin is the answer. Thanks!!

[mykelk]

Strange! But very good to know!