It's just few lines of code but it works as it should. There was no information in the course on how to implement UCB1 with Gaussian Process, so I had to get a little bit creative. Instead of the classical condifence bound of UCB1, I used the 96% confidence interval of the Gaussian Process (the upper part obviously). We already have the empirical mean and we sum the confidence bounds computed directly by the GP. This way we have an upper bound that decreases the more samples we get in point X (as it should be) but at the same time we have the advantage of the continuity of the GP, therefore confdence bounds of close points will influence one another.
It's just few lines of code but it works as it should. There was no information in the course on how to implement UCB1 with Gaussian Process, so I had to get a little bit creative. Instead of the classical condifence bound of UCB1, I used the 96% confidence interval of the Gaussian Process (the upper part obviously). We already have the empirical mean and we sum the confidence bounds computed directly by the GP. This way we have an upper bound that decreases the more samples we get in point X (as it should be) but at the same time we have the advantage of the continuity of the GP, therefore confdence bounds of close points will influence one another.