unexpected behaviour in LowPrev set_lower vs. __setitem__

[equaeghe]

Documented, but it still bit me:

>>> ev = lambda event: Event(pspace, event)
>>> print(lprob)
a b     : 1/2
a   c   : 1/2
a     d : 1/2
  b c   : 1/2
  b   d : 1/2
    c d : 1/2
a b c   : 1/2
a b   d : 1/2
a   c d : 1/2
  b c d : 1/2
>>> lprob.set_lower(ev('bc'), 0)
>>> print(lprob)
a b     : 1/2
a   c   : 1/2
a     d : 1/2
  b c   : 1/2
  b   d : 1/2
    c d : 1/2
a b c   : 1/2
a b   d : 1/2
a   c d : 1/2
  b c d : 1/2
>>> lprob[ev('bc'), True] = (0, None)
>>> print(lprob)
a b     : 1/2
a   c   : 1/2
a     d : 1/2
  b c   : 0
  b   d : 1/2
    c d : 1/2
a b c   : 1/2
a b   d : 1/2
a   c d : 1/2
  b c d : 1/2

I don't know whether anything should be done, but I wanted to let you know it may cause confusion.

[mcmtroffaes]

Yep - there are use cases for both. Perhaps the documentation is too cryptic about it? A warning in the docs seems appropriate.

[equaeghe]

Or perhaps introduce reset_lower methods (and similar), then it will also become clear from the fact that there are two; and/or giving another name to set_lower, constrain_lower. Just ideas... perhaps best to let them linger for a while before deciding (not) to act upon them. (Unless you don't like open issues ;-).)

[mcmtroffaes]

... and another commit which relates to this:

https://github.com/mcmtroffaes/improb/commit/26626fbf6a84b126c2b43bdb3e780bd8dc2233a7

This is just a proof-of-concept, nothing of this is in the master yet. The essential idea is to simply the interface by letting set_lower being the same as setitem. set_upper is a shorthand for set_lower(-gamble, -uprev). The same symmetry is used for the get_lower and get_upper functions.

Comments welcome (particularly if you think that this is a bad idea... :-) ).

[equaeghe]

I think this (commit https://github.com/mcmtroffaes/improb/commit/0f1dbf707437e6a7bbac3a33115ed1456b2745dd) is a good idea (partly due to taste), but it is a big change. Simplifying the internal representation will probably lighten some work. On the other hand, getting out probability intervals and prevision intervals out will require more work (not necessarily computational, but rather program code)?

I'm not to clear on the impact of commit https://github.com/mcmtroffaes/improb/commit/26626fbf6a84b126c2b43bdb3e780bd8dc2233a7 -- you're working towards a 'unique data structure' that does it all? (Or two: vertex-representation and halfplane-representation?)

[mcmtroffaes]

https://github.com/mcmtroffaes/improb/commit/0f1dbf707437e6a7bbac3a33115ed1456b2745dd is indeed a big change - already in this experiment, it removes a lot of bloat, so I think I'll go ahead with it (main downside is that it requires a change to the public interface). The str function could still report intervals if it detects the negated gamble, or complementary event, in the domain. repr can report the actual list of assessments.

The reason for https://github.com/mcmtroffaes/improb/commit/26626fbf6a84b126c2b43bdb3e780bd8dc2233a7 is that I've come to realize that all of the algorithms make sense also for general lower previsions, as approximations to the natural extension. Once implemented as such, there is no reason for these special classes to remain in existence. I've never liked them anyway (they lead to circular imports etc. which is almost always a sign of flawed design). The interface will also gain simplicity as a result (i.e. no more need to choose which class to use for your application).

If you are suggesting also to derive a CredalSet class from LowPrev, as a counterpart to LowPoly, storing full conditional probabilities (in Dubins's sense) instead of constraints, then I agree that this makes plenty of sense. The main issues are 1) to find a good representation of full conditional probabilities (first choice that comes to my mind is to use a list of probability measures, one for each zero layer), and 2) to identify an algorithm to find this set for an arbitrary LowPoly and vice versa. But this is obviously out of scope for this ticket. :-)

[equaeghe]

As regards what str/repr should/could output: it might be useful to have a canonical 'minimal' representation (e.g., marginal gambles or (f-min{f})/range{f}-normed gambles and their prevision) that is used internally - this may speed up natural extension in some cases, as it minimizes the number of gambles that it considers (i.e., instead of having both P(f)=1 and P(2f)=2); other known prevision values may be put in a cache for quick lookup. repr would show this canonical representation, str would pretty-print it, and other methods could give other output. Before taking this idea seriously, you'd have to answer the question how important it is (to you) to preserve the original non-canonical input/assessment.

As regards CredalSet: yes, I was thinking about dual representations. I'd use probability mass functions instead of measures, but you might have meant that. Has work been done on (conditioning) credal sets of full conditional measures?

[mcmtroffaes]

Switching to a canonical representation makes a lot of sense; using marginally desirable gambles for internal representation simplifies the code. The main potential issue is that someone might write:

lpr[f,True] = 0
...
lpr[2f,True] = -1

If using marginally desirable gambles, we can do one of three things here: ignore the last statement (as -1/2 < 0), or translate it into lpr[f, True] = -1/2, or throw a ValueError. Perhaps the first option is most reasonable, but then, what with:

lpr[f, True] = 0
...
lpr[f, True] = -1

I think that the least surprising thing is just to store the constraints as given without further modification. What you ask for, with sets of desirable gambles, requires a different interface, which avoids these issues (i.e. a DesirableGambles class, which would be a collections.MutableSet rather than a collections.MutableMapping ;-) ).

CredalSet: indeed, I meant (full conditional) probability mass function. Work on these is very limited, although Paolo Vicig and Renato Pelessoni have looked at this in some detail. It might be possible to use the algorithm for checking ASL in Section 3.2 of their IPMU'98 paper http://www2.units.it/renatop/papers/consistency_problem_preliminary.pdf to sequentially construct all extreme full conditional probability mass functions. The work of Angelo Gilio could also be useful (but I haven't checked this in detail yet). There is possibly more work that I'm not aware of. In any case, it's clear that it won't be obvious.