Clarify sample continuity theorem

[chrisyeh96]

I'm trying to understand the Gaussian process sample continuity theorem stated on page 35. The text says:

The following result is adequate for most settings arising in practice and may be proven as a corollary to the slightly weaker (and slightly more complicated) conditions assumed in Adler and Taylor’s theorem 1.4.1.

Would it be possible to provide any additional guidance on how to derive the theorem stated on p.35 of the Bayes Opt book from Adler and Taylor's Theorem 1.4.1? I tried reading Adler and Taylor's Theorem, but I struggled to understand it.

[bayesoptbook]

I'm sorry for the confusion. The key to this argument could, perhaps with some work, have been squeezed into a footnote (and I may try to do so now).

We begin with the expression on the LHS of Adler and Taylor's sufficient condition (their (1.4.4)), which, by no coincidence, was also discussed in #47:

E|ϕ - ϕ'|².

Let us assume that K is α-Hölder continuous and show that Adler and Taylor's condition is satisfied. Let's ignore the mean function for now and assume the process is centered, μ ≡ 0 [*].

By assumption, we have

E|ϕ - ϕ'|² = K(x, x) + K(x', x') - 2K(x, x') ≤ C|x - x'|ᵃ

for some C ≥ 0. Consider the function

h(d) = dᵃ |log d|¹⁺ᵃ

on the (compact) interval [0, 1]. Note that h(0) = h(1) = 0 on the boundary and that h > 0 on the interior (0, 1). Thus h achieves a maximum M > 0 (depending on alpha) somewhere in (0, 1). That implies

E|ϕ - ϕ'|² ≤ C|x - x'|ᵃ ≤ CM / |log |x - x'||¹⁺ᵃ

for |x - x'| < 1. That is sufficient for Adler and Taylor's condition.

[*] As before, to deal with nonzero μ just assume it's continuous and add it back in.

[bayesoptbook]

Did this help @chrisyeh96?