search

olethrosdc / beliefbox

Automatically exported from code.google.com/p/beliefbox
4 stars 5 forks source link

Testing out UCRL algorithm does not work #1

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Download beliefbox and compile subdirectories
2. ./bin/online_algorithms --n_runs 2 --environment RiverSwim --gamma 1 
--n_steps 1000 --epsilon 0.0 --algorithm UCRL  > UCRLout

What is the expected output? What do you see instead?
Instead of text file UCRLout filling with information, as per the same commands 
with QLearning algorithm, it just stays blank and does not populate the text 
file even after being left for an hour.

What version of the product are you using? On what operating system?
beliefbox r-943 on Max OS X

Please provide any additional information below.
Thanks for the toolbox, it seems great, hope I can get it working!

Original issue reported on code.google.com by Ian.Osb...@gmail.com on 12 May 2013 at 6:16

olethrosdc commented 8 years ago

Hi. I think the problem is my implementation of OptimisticValueIteration. To make it stable for gamma = 1, you need to check some alternative convergence condition. There are also a couple of other ways to improve it: normally to get the tightest possible bound on the transitions you need to solve a linear program.

If you can help fixing it, it'd be nice.