search

BrainsOnBoard / bob_robotics

Collection of code for interfacing with robot platforms + simulations and visualisation
GNU General Public License v2.0
2 stars 6 forks source link

``Navigation::InfoMax`` currently silently ignores the specified image mask #221

Open alexdewar opened 3 years ago

alexdewar commented 3 years ago

This is potentially a problem for e.g. the snapshot_bot code as we are tacitly assuming that the mask is being applied. Note that, if anything, it makes the problem harder for the algorithm, so we shouldn't worry that we've been "cheating".

The fix is conceptually very easy (we just need to select only the unmasked pixels for training/testing images), so I'll see if I can sort it while I'm refactoring the navigation code.

alexdewar commented 3 years ago

Tagging people FYI: @tnowotny @aphilippides @neworderofjamie @tenxlenx

NB this is the code we used in e.g. the Alife paper.

neworderofjamie commented 3 years ago

I may well be wrong but I don't think this is correct. Wouldn't that be equivalent to providing a black input to the masked pixels (which isn't the same thing)?

alexdewar commented 3 years ago

Wait, you mean the fix (a06fdc1)?

In the end, the way I did it was to select only the unmasked pixels as inputs and reduced the number of inputs to the network accordingly. I think this is the right way to do it.

alexdewar commented 3 years ago

Wait, you mean the fix (a06fdc1)?

In the end, the way I did it was to select only the unmasked pixels as inputs and reduced the number of inputs to the network accordingly. I think this is the right way to do it.

Ok, on reflection it isn't :laughing:. It gets complicated when a masked image is being rotated (e.g. with an RIDF).

I suppose we probably ought to additionally zero out the masked pixels in the current view. I think we should still drop the inputs to the weights matrix corresponding to the masked pixels in the unrotated mask (as by definition the snapshots aren't providing inputs here).

What do you think, @aphilippides ?

alexdewar commented 3 years ago

More annoyingly, we have issue #222

tnowotny commented 3 years ago

would it be worth to get together when Jamie is back and have an in-depth discussion about this (I am not sure I can make any meaningful contributions without some prepping from you guys)?

alexdewar commented 3 years ago

It's probably not worth a whole meeting by itself but it'd be worth having a chat about I think.

tnowotny commented 3 years ago

we should get together anyway to take stock of some things and plan the summer - but we should wait until Jamie is back. Next Wed or Friday I would suggest.

alexdewar commented 3 years ago

Good plan.