Open gautams3 opened 4 years ago
if n=0 and ltn = 0, the sqrt(ltn/n) calculation will produce a NaN. Can you think of any other way to handle this?
It seems like instead of producing a NaN, we are assigning 0/0 a value of 0. Assigning 0 to undefined expression 0/0 does help in disabling UCB for all children nodes (even those with n != 0
).
One failing edge case is pure exploration (completely random policy) where exploration coefficient coeff_c = Inf
. Consider this scenario:
total_n = 1
and ltn = 0
. So there is some child node ch
with nch = 1
. Its criterion value is
criterion_value_ch = value_ch + coeff_c * sqrt(0/1)
coeff_c * sqrt(0/1) = Inf * 0 = NaN
. All other nodes might have finite critertion values, and comparing NaNs to these might result in an error.
The easy solution to this is to disable UCB for all children nodes if ltn=0. This way we avoid the Inf*0
expression. The code can read
if ltn <= 0.0 # ltn<0 implies all n = 0, ltn = 0 imples 1 n = 0. Both cases covered here.
criterion_value = t.v[node]
elseif
.
.
.
end
If this edge case is important enough to address, I can send over a PR.
I am not sure of the logic behind adding
ltn<= 0.0
in the simulate() function. It implies that for two first two tree queries (ltn
=-Inf
and0
), it uses the node value as criterion. I understand that we use node value whenltn = -Inf
, but don't understand why it is the case forltn = 0
Ref this line in simulate()