Given a stream of participants, choose the stream that
Want the scoring function itself to be independent of the algorithm used to assign points.
For now we can try a density comparison algorithm:
Density comparison
Define a function over the space we want to sample: maybe uniform or otherwise
Take a KDE of the points we sampled, this gives us
Compare the KDE to the desired density function at a number of points
Integrate difference over the spaceusing a monte carlo integration (or a grid)?
A gaussian kernel minimizes the number of assumptions (Statistical Rethinking page 403)
We can start with a fairly wide kernel, assuming that the landscape will be reasonably flat in any given dimension (ie, likely to be linear or u-shaped in any one dimension)
As we take data, we can look at a different kernel size, which might suggest a different algorithm.
Given a stream of participants, choose the stream that
Want the scoring function itself to be independent of the algorithm used to assign points.
For now we can try a density comparison algorithm:
Density comparison