search

swiseman / bethe-min

7 stars 2 forks source link

not aligned computation of bethe energy #2

Open FirstHandScientist opened 4 years ago

FirstHandScientist commented 4 years ago

Explanation would be appreciated:

https://github.com/swiseman/bethe-min/blob/afd9733932a50454a32a612751245775a5c5766e/ising.py#L383

In the computation of bethe_unary, why do unary0 and unary1 get minused? This minus operation is not present in definition of Bethe free energy.

yoonkim commented 4 years ago

Hi! I think there are various parameterizations of the bethe-free energy which end up being equivalent.

Below I tested the log partition calculated from variable elimination vs log partition calculated from bethe free energy (whose input are the exact unary/binary marginals), and they seem be close enough (although of course they arent supposed to match exactly since for non-tree models bethe free energy does not equal the log partition)

for n in [3, 4, 5, 6, 7, 8, 9, 10]:
    ising = ising_models.Ising(n)
    log_Z_ve = ising.log_partition_ve()
    unary_marginals_ve, binary_marginals_ve = ising.marginals()
    log_Z_bethe = -ising.bethe_energy(unary_marginals_ve, binary_marginals_ve)
    print(log_Z_ve.item(), log_Z_bethe.item())
FirstHandScientist commented 4 years ago

Hi, thanks for the replay. I agree that the partition function aligns.

What I feel confused is the defined and usage of Bethe free energy (BFE): In the code on computation of BFE, it seems that the definition in Section 3.3 (equation 33) in Yedidia [67] is used.

The BFE definition (equation 1) in your paper, seems to be based on Section 3.2 of Yedidia[67]. But it seems to have one extra term than the combination of average energy (29) and entropy (31) in Section 3.2 of Yedidia[67]. Since factor alpha can be unary and binary factors in your paper, aren't the entropy part also sum the unary factors when alpha is unary factors in your BEF (1) ?