Distributed decision making

[Alex-Linhares]

Here's a quote from some programming book:

Every time your code references global data, it ties itself to the other components that share that data. Even globals that you intend only to read can lead to trouble (for example, if you suddenly need to change your code to be multithreaded). etc etc...

Now, please take a look at the reasoning of pages 41 and 42 of this paper:

https://github.com/Alex-Linhares/FARGonautica/blob/master/Literature/Chess-Capyblanca-2014-Linhares-Information%20Sciences.pdf

Why are we using Temperature as the basis for decision-making? Even as temperature is a wonderful metaphor, with many historical reasons for its adoptance, can't we try to move to entropy? That is, can't we try out entropy measures based on the unhappiness and the relevance of particular structures, instead of having a big-brotheresque variable T that everyone has to check in order to make a decision.

Why would it be interesting?

• Because the system becomes more distributed and more psychologically (and neuroscientifically) plausible.
• Because in cases such as the "abc:abd xyz:?" snag.... as T goes up, the entire construction is up for grabs and reconfiguration... perhaps some structure can, and should, remain intact, and only the very unhappy objects should have a high probability of being destroyed.
• It preserves parallel terraced scan and it preserves the thermodynamics metaphor (the brain is, after all, using energy).

[LSaldyt]

I've further documented the usages of temperature in the Python repository.

I've also documented the usages of temperature in the original (LISP) version of copycat, by Melanie Mitchell. This is probably relevant because changes to the temperature calculations may have been lost in translation (I believe that the Python version of the repository was translated from the Java version, which was translated from the original LISP).

Next, I'm going to come up with metrics to analyze the distribution of answers, and then experiment with some different algorithms for temperature and then entropy.

P.S. To the best of my knowledge, there is nothing preventing safely reading global constants in a multi-threaded program, or replicating global constants in a psychologically/biologically plausible way, which is to say the a problem with temperature is that it is mutable and global, not just global.

[LSaldyt]

The original LISP documentation for (getAdjustedProbability):

This function is a filter: it inputs a value (from 0 to 100)[sic: from 0 to 1] and returns a probability (from 0 - 1) based on that value and the temperature. When the temperature is 0, the result is (/ value 100)[sic: value], but at higher temperatures, values below 50 [sic:.5] get raised and values above 50[sic:.5] get lowered as a function of temperature. I think this whole formula could probably be simplified.

So, we need a formula that makes unlikely options more likely and likely options less likely as the temperature increases.

I've tested out several formulas in a spreadsheet, using coloring so that I can see how they behave at different temperatures. I'm not sure why the entropy formula is successful in providing diversity of answers. I'm also not sure how well the original copycat function was working either, as it seemed unnecessarily complicated.

The formula I built that makes the most intuitive sense to me is (T/100)(1-p) + ((100-T)/100)*p. This will move probabilities towards their inverses. For example, at temperature 100, a probability of 1 becomes a probability of 0 and a probability of 0 becomes a probability of 1. In between is a gradient effect, where all probabilities become .5 at T=50. This is easier to see with the colored spreadsheet. I feel that this formula or some weighted adjustment of it will be the most effective in providing better functionality than the original formula.

In the original code:

        if value == 0 or value == 0.5 or self.value() == 0:
            return value
        if value < 0.5:
            return 1.0 - self.getAdjustedProbability(1.0 - value)
        coldness = 100.0 - self.value()
        a = math.sqrt(coldness)
        c = (10 - a) / 100
        f = (c + 1) * value
        # return max(f, 0.5)
        # return max(f, 0.0)
        # return (0 + (-f * math.log2(f)))
        return -f * math.log2(f)

c + 1 acts as a scalar, moving between 1.0 and 1.1 as Temperature increases. So, f is a scaled version of the original probability. (Unchanged at T=0 and slightly (10%) increased at T=100). The original max(f, 0.5) indicates the desire to curve lower probabilities higher and higher probabilities lower.

[LSaldyt]

The next step will likely be to statistically analyze the answer distributions of different getAdjustedProbability functions. Your (@Alex-Linhares) comments indicate that you've done some statistical tests. What tests did you use?

[Alex-Linhares]

haha I just googled some statistical significance online webpage and threw in the numbers... Guess we're going to have to be more pro than that.

I'm studying your analysis of the temp, original and new.

[Alex-Linhares]

oh, good argument.. temp is also mutable in a real parallel system...

[LSaldyt]

I've updated get-adjusted-probability to the following formula:

curved = (
(T/100) * .5 + 
(100-T)/100 * (p ** 1.05 if p < 0.5 else p ** (1/1.05)))

I believe this to be accurate to Mitchell's intentions and more elegant than the original formula. However, there are many other improvements (to other features) that will improve the answer distributions. The value of 1.05 can be tweaked to change the smoothness of the adjustment. 2 is generally a pretty good value as well, and is used below to compare to the original formula. A fork using this formula is available here.

A comparison of the original and "best" formulas on a few copycat problems (I've omitted abc:abd::xyz:_ because it is affected by too many other issues in copycat): dists.txt