Defining distributions with scalars but passing list to log prob (Negative Binomial in particular)

[ThomasHeap]

If I want to do something like:

dist = alan.NegativeBinomial(total_count=5, probs=0.5)
samp = [0,1,2,3,4,5]
print(dist.log_prob(samp))
# Expecting output of length 5

It will presently throw:

https://github.com/alan-ppl/alan/blob/997a181954f44ea8d437ac9fd80220478c08808c/alan/dist.py#L141

Which seems wrong, perhaps the assert should be changed to:

assert x.ndim == self.result_ndim + self.unnamed_batch_dims  or x.ndim == self.result_ndim + self.unnamed_batch_dims + 1

[LaurenceA]

try

samp = t.tensor([0.,1,2,3,4,5])

On Wed, Mar 8, 2023 at 5:23 PM thomas @.***> wrote:

If I want to do something like:

dist = alan.NegativeBinomial(total_count=5, probs=0.5)samp = [0,1,2,3,4,5]print(dist.log_prob(samp))# Expecting output of length 5

It will presently throw:

https://github.com/alan-ppl/alan/blob/997a181954f44ea8d437ac9fd80220478c08808c/alan/dist.py#L141

Which seems wrong, perhaps the assert should be changed to:

assert x.ndim == self.result_ndim + self.unnamed_batch_dims + 1

— Reply to this email directly, view it on GitHub https://github.com/alan-ppl/alan/issues/9, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABAUSQVJBZ3IA47RKEI3BMDW3C57PANCNFSM6AAAAAAVUBJCFQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[LaurenceA]

Even that doesn't work, but to me that's expected behaviour.

If you want a valid distribution over 6 things, you need:

dist = alan.NegativeBinomial(total_count=5*t.ones(6), probs=0.5)

samp = t.tensor([0.,1,2,3,4,5])

On Thu, Mar 9, 2023 at 9:15 AM Laurence Aitchison < @.***> wrote:

try

samp = t.tensor([0.,1,2,3,4,5])

On Wed, Mar 8, 2023 at 5:23 PM thomas @.***> wrote:

If I want to do something like:

dist = alan.NegativeBinomial(total_count=5, probs=0.5)samp = [0,1,2,3,4,5]print(dist.log_prob(samp))# Expecting output of length 5

It will presently throw:

https://github.com/alan-ppl/alan/blob/997a181954f44ea8d437ac9fd80220478c08808c/alan/dist.py#L141

Which seems wrong, perhaps the assert should be changed to:

assert x.ndim == self.result_ndim + self.unnamed_batch_dims + 1

— Reply to this email directly, view it on GitHub https://github.com/alan-ppl/alan/issues/9, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABAUSQVJBZ3IA47RKEI3BMDW3C57PANCNFSM6AAAAAAVUBJCFQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[ThomasHeap]

I'd like it to return a vector of log probabilities the same size as samp. I'm not trying to have a distribution over 6 things I've got 6 samples from a distribution over one thing.

If I do

dist = alan.NegativeBinomial(total_count=5*t.ones(6), probs=0.5)
samp = t.tensor([0.,1,2,3,4,5])
print(dist.log_prob(samp))

It returns a scalar log probability.

Turning the list to a tensor also doesn't solve the problem either.

[LaurenceA]

That's expected behaviour.

If you want a valid distribution over 6 things, you need:

dist = alan.NegativeBinomial(total_count=5*t.ones(6), probs=0.5)

samp = t.tensor([0.,1,2,3,4,5])

Or you can add a torchdim to samp. And it should be returning a scalar (which makes things more convenient downstream).

On Thu, Mar 9, 2023 at 9:35 AM thomas @.***> wrote:

I'd like it to return a vector of log probabilities the same size as samp. I'm not trying to have a distribution over 6 things I've got 6 samples from a distribution over one thing.

If I do

dist = alan.NegativeBinomial(total_count=5*t.ones(6), probs=0.5)samp = t.tensor([0.,1,2,3,4,5])print(dist.log_prob(samp))

It returns a scalar log probability.

Turning the list to a tensor also doesn't solve the problem either.

— Reply to this email directly, view it on GitHub https://github.com/alan-ppl/alan/issues/9#issuecomment-1461661791, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABAUSQWUL7ODFP6SIM5HH33W3GP7TANCNFSM6AAAAAAVUBJCFQ . You are receiving this because you commented.Message ID: @.***>

[ThomasHeap]

OK well do I have to add a dimension to samp if I want to get a log probability for each of the entries of samp?

[LaurenceA]

Yes ... but why is all this such an issue?

On Thu, Mar 9, 2023 at 9:41 AM thomas @.***> wrote:

OK well do I have to add a dimension to samp if I want to get a log probability for each of the entries of samp?

— Reply to this email directly, view it on GitHub https://github.com/alan-ppl/alan/issues/9#issuecomment-1461671634, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABAUSQSZCNS2BBDVHZ72WBTW3GQVVANCNFSM6AAAAAAVUBJCFQ . You are receiving this because you commented.Message ID: @.***>

[ThomasHeap]

Its inconsistent with torch distributions. I think if I have two scalar parameters and I pass a vector of samples of length 6 then I'd expect to get a vector of log probabilities also of length 6.

[LaurenceA]

We don't care. dist.py is not supposed to be used outside Alan. dist.py is adapted to make everything downstream in Alan as easy as possible.

On Thu, Mar 9, 2023 at 10:18 AM thomas @.***> wrote:

Its inconsistent with torch distributions. I think if I have two scalar parameters and I pass a vector of samples of length 6 then I'd expect to get a vector of log probabilities also of length 6.

— Reply to this email directly, view it on GitHub https://github.com/alan-ppl/alan/issues/9#issuecomment-1461736009, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABAUSQU5GGTHOOK56CHHLWDW3GVA7ANCNFSM6AAAAAAVUBJCFQ . You are receiving this because you commented.Message ID: @.***>

[ThomasHeap]

Right, ok I'll close this.