Closed thomasgudjonwright closed 3 years ago
(spoken to Tom offline about this)
To expand on what Tom is saying: there are two obvious conventions you could adopt for the ordering of your inputs to a multi-output GP. Suppose that you have P
outputs and N
"inputs", meaning that your complete input is of length NP
. You can either
P
blocks of N
(MOInput), orN
blocks of P
(the one we're not currently doing)Invenia's code currently utilises the latter, while JuliaGaussianProcesses presently only has support for the former.
We should support both conventions because
ColVecs
and RowVecs
are both necessary.I don't think this really blocks the current PR, but it'll be blocking for the next one.
I quite like BlockByOutputs
and BlockByInputs
, but since these are both collections of inputs (in our vocab), I wonder whether now is the right time to also solve the "inputs" vs "inputs" naming issue.
i.e. for
x = randn(2)
y = MOInput(x, 3)
both are valid "inputs", but it would be helpful to be able to distinguish between x
and y
when referring to inputs.
I wonder whether we should call y
"inputs" and x
"locations". We could then say things like:
"An input to a multi-ouptut GP comprises a location x
and output index p
"
as opposed to having to say
"An input to a multi-output GP comprises an input x
and a output index p
".
Under this naming convention and Tom's proposal, BlockByOutputs
and BlockByLocations
would be the obvious names.
Instead of location, we could use more classical terms like feature vector or input sample.
About the whole ordering shenanigans, this would also be easier if we considered 4-tensors instead :trollface:
Well it might make some specific things easier (I'm actually not sure what things it would make easier), but it would make lots of things harder, including
kernelmatrix
from the ordering of the inputs.kernelmatrix
doesn't output an AbstractMatrix
anymore. The beauty of the current approach is that one can write stuff on top of KernelFunctions
and assume that, regardless the kernel you pass in, you'll always get the same kind of thing back, and it's completely predictable.kernelmatrix(k, x, y)[p, q] == k(x[p], y[q])
.It really just comes down to the fact that we would be privelidging multi-output GPs, which I think is fundamentally the wrong way to go about this. IMHO it's really important for our ability to easily build stuff on top of KernelFunctions / AbstractGPs that we don't have to think too hard about whether a particular GP is a single-output or multi-output, unless you're doing something multi-output-specific.
edit: to be fair, we should probably lead with that in the docs. I've added this to my todo list.
Instead of location, we could use more classical terms like feature vector or input sample.
These also seem like reasonable proposals -- I quite like feature vector, although it's perhaps a bit verbose.
Perhaps just feature and BlockByFeatures
? It would be shorter and features don't have to be vectors necessarily.
I can't seem to assign myself to this, can I only do so on my fork?
I think you need to be a member of the organization, which can be solved of course
How about IsotopicByFeatures
and IsotopicByOutputs
? [1] (section 3.1) suggests that Isotopic
is the term used in the geostatistics literature for situations in which all of the outputs have the same set of inputs. This is perhaps an improvement on Blocked
/ Block
because it people won't know what it means, and will have to look it up. Conversely, Block
could mean any number of things.
[1] - https://arxiv.org/pdf/1106.6251.pdf
edit: then a super-type for the two could be something like AbstractIsotopicVector
or something.
The idea would be to introduce two new types (names open for discussion), lets call them
BlockByInputs
andBlockByOutputs
for the purpose of this issue.BlockByOutputs
would be equivalent to the currentMOInput
, ie, given and inputx = [4.5, 5.5]
, we have:Conversely,
BlockByInputs
would look something like this for the same input:The reasoning for this is that each will format the
kernelmatrix
differently. TheLMM
expects a kernelmatrix that would be constructed properly using the latter (also works by usingcollect(MOInput(x,2))
as input tokernelmatrix
).