linear combinations as training inputs

[RPrudden]

I'm interested in a situation where my training inputs are not at specific locations, but are instead linear combinations over locations. It's possible to work with this by using the explicit matrix algebra, but I'd prefer to use GPyTorch for larger examples.

As far as I can tell, the implementation of models such as ExactGP relies on training inputs being point locations. Is this correct? If so, do you have any insight on how I could go about extending it to linear combinations, and how difficult this is likely to be?

Thanks in advance!

[jacobrgardner]

Are these fixed linear combinations of some basis or learned? If fixed you can just set your train data to AX where A is the basis; if learned you can do the same but make A your training data and X a Parameter perhaps?

Or is your point you'd like to avoid the explicit computation of AX

[RPrudden]

Yes, they are fixed linear combinations. I think the main issue is that I want to provide training data in the space of AX, but get a posterior distribution in the space of X.

[jacobrgardner]

Do you have a reference for how this is done normally? It seems to me that in general the effect of A is going to be kernel dependent, and a little hard to deal with in terms of getting a posterior for X with the exception of a linear kernel. For example, for the RBF kernel you could view A as replacing the Euclidean distance with the Mahalanobis distance ||x - x'||_{M} where M = A^T A.

I generally believe it might be possible, but it's actually not immediately obvious to me how we would provide training data in the span of A but get a posterior in the span of I (again with the exception of the linear kernel).