ENH: adaptive scans

[CJ-Wright]

Perform adaptive scans with live/autonomous parameter tuning.

Expected Behavior

Experimental parameters tune themselves to get the "best" data or focus on the "most interesting" data.

Current Behavior

All parameters are human tuned, with unknown/questionable metrics

Possible Solution

1. Analysis pipeline writes new RunEngine messages based off of data analysis
2. Pipeline passes messages back to RunEngine
3. RunEngine, which was awaiting the messages, executes the message.

Context

1. Save users tons of time tuning things
2. May tune better than users can
3. Tuning will be reproducible
4. Awesome new automated science

[sbillinge]

This is a "save it for later"

[CJ-Wright]

Signature for factory, maybe?

Acq

# This is for continuous variables (not the filters)
def adaptive_factory(mover1, lower1, upper1, mover2, lower2, upper2, ..., per_step=None, **kwargs):
    def inner():
        extra_md = {'adapt': {k: {'lower': bla, 'upper': bla2} for k in movers}}
        # issue extra_md into start doc
        yield from open_run(...)
        next_step = make_message(read_from_place())
        yield from next_step
    return inner()

An

1. Select autonomous pipeline based on start doc information
2. Stream documents in
3. Handle bootstrap case (no prior knowledge, maybe latin hypercube?)
4. If not bootstrap use kriging to find new points (infill)
5. Write commands to file
6. Repeat

Note that we can get around the serialize an ophyd object issue by having a dict shared between the two processes which maps string names to ophyd objects. This will allow make_message to read the file (which consists of a name an a position) and then make a working message (with an ophyd object) to send to the run engine.

[CJ-Wright]

The above comment addresses the case where there is a one to one relationship between the pipeline and the acq. We have another use case where the acq keeps running until the pipeline says stop. In this case we want to keep pinging the detector while awaiting the data from the pipeline. I'm not certain how to set that up.