Create "flyscanner" for putting top-off timestamps in as events

[stuwilkins]

IOC is ready and written by @stuwilkins, needs ophyd/bluesky object.

[danielballan]

What's the PV?

[stuwilkins]

https://github.com/NSLS-II-CSX/timestamp https://gist.github.com/stuwilkins/5b47bf14238bad548a79

Attn @cmazzoli @ambarb

[danielballan]

attn @tacaswell @dchabot @klauer This is a blocking issue, so to speak, for some testing at CSX. I said we would to attempt to deliver this by Friday, October 30.

[danielballan]

It might be as simple as this, @stuwilkins:

import time as ttime
import epics

class WaveformCollector:
    def __init__(self, name, pv_name):
        self._pv_name = pv_name
        self._name = name
        self._pv = epics.PV(pv_name, auto_monitor=True)

    def kickoff(self):
        # nothing to do?
        pass

    def collect(self):
        payload = self._pv.get()
        for v in payload:
            # yield *partial* event documents, finished up by the RunEngine
            ev = {'data': {self._name: v},
                 'timestamps': {self._name: v},
                 'time': ttime.time()}
            yield ev

    def stop(self):
        # nothing to do, unless there is some way of "clearing" the pv
        pass

    def describe(self):
        return [{self._name: {'source': self._pv_name, 'dtype': 'number', 'shape': None}]

exposure_time_flyer = WaveformCollector('exposure_time', 'XF:23ID1-CT{TS-TEST}Val:Time-Wfrm')
ct.flyers = [exposure_time_flyer]
ct()

[klauer]

@danielballan Something like that should work. Would recommend against auto_monitor for such a large array, beside for it being completely unnecessary in this case.

Also, does it not make sense to have the timestamp of the event be the one retrieved from the EVR?

[stuwilkins]

Hi ken,

The problem with not using auto_monitor is that it doesn’t allow the use of the NORD or NELM fields which makes it much more difficult… Have look at:

http://cars9.uchicago.edu/software/python/pyepics3/arrays.html#variable-length-arrays-nord-and-nelm

I could not get this to work (and properly truncate the array) unless auto_monitor was on.

S

From: K Lauer notifications@github.com<mailto:notifications@github.com> Reply-To: NSLS-II/wishlist reply@reply.github.com<mailto:reply@reply.github.com> Date: Friday, October 23, 2015 at 2:01 PM To: NSLS-II/wishlist wishlist@noreply.github.com<mailto:wishlist@noreply.github.com> Cc: Stuart Wilkins swilkins@bnl.gov<mailto:swilkins@bnl.gov> Subject: Re: [wishlist] Create "flyscanner" for putting top-off timestamps in as events (#76)

@danielballanhttps://github.com/danielballan Something like that should work. Would recommend against auto_monitor for such a large array, beside for it being completely unnecessary in this case.

Also, does it not make sense to have the timestamp of the event be the one retrieved from the EVR?

— Reply to this email directly or view it on GitHubhttps://github.com/NSLS-II/wishlist/issues/76#issuecomment-150648601.

[stuwilkins]

Also, don’t we want to add N events when the array in N length. I.e. each top-off injection should be one event.

S On Oct 23, 2015, at 2:01 PM, K Lauer notifications@github.com<mailto:notifications@github.com> wrote:

@danielballanhttps://github.com/danielballan Something like that should work. Would recommend against auto_monitor for such a large array, beside for it being completely unnecessary in this case.

Also, does it not make sense to have the timestamp of the event be the one retrieved from the EVR?

— Reply to this email directly or view it on GitHubhttps://github.com/NSLS-II/wishlist/issues/76#issuecomment-150648601.

[klauer]

@stuwilkins If that's the case, I'd strongly recommend trying to fix it at the source (it's a pyepics thing not an EPICS thing, right?) or come up with an alternative. Monitoring these massive arrays is just bound to cause problems somewhere down the line. Consider instead:

In [1]: pv = epics.PV('XF:31IDA-BI{Cam:Tbl}test', auto_monitor=False)

In [2]: nord = epics.PV('XF:31IDA-BI{Cam:Tbl}test.NORD')

In [3]: pv.get(count=int(nord.get()))
Out[3]: array([2, 3, 4, 5], dtype=int16)

In [4]: pv.get()
Out[4]: array([2, 3, 4, ..., 0, 0, 0], dtype=int16)

[danielballan]

Yes, I think we all agree that there should be one Event per reading. But the array only needs to be ready once, at the end, where is will be chopped up into Events by the collect method, as written. On Fri, Oct 23, 2015 at 2:43 PM K Lauer notifications@github.com wrote:

@stuwilkins https://github.com/stuwilkins If that's the case, I'd strongly recommend trying to fix it at the source (it's a pyepics thing not an EPICS thing, right?) or come up with an alternative. Monitoring these massive arrays is just bound to cause problems somewhere down the line. Consider instead:

In [1]: pv = epics.PV('XF:31IDA-BI{Cam:Tbl}test', auto_monitor=False)

In [2]: nord = epics.PV('XF:31IDA-BI{Cam:Tbl}test.NORD')

In [3]: pv.get(count=int(nord.get())) Out[3]: array([2, 3, 4, 5], dtype=int16)

In [4]: pv.get() Out[4]: array([2, 3, 4, ..., 0, 0, 0], dtype=int16)

— Reply to this email directly or view it on GitHub https://github.com/NSLS-II/wishlist/issues/76#issuecomment-150659941.

[stuwilkins]

@klauer You are right, I have implemented as you suggested works fine.

@danielballan, I am trying this now but it seems that kickoff is expected to return something ...

[tacaswell]

Kick off needs to return a status object.

[stuwilkins]

The following code is found to work.....

import time as ttime
import epics
import numpy as np

class WaveformCollector:
    def __init__(self, name, pv_basename, data_is_time=True):
        self._name = name
        self._pv_basename = pv_basename
        self._pv_sel = epics.PV("{}Sw-Sel".format(pv_basename))
        self._pv_rst = epics.PV("{}Rst-Sel".format(pv_basename))
        self._pv_wfrm_n = epics.PV("{}Val:TimeN-I".format(pv_basename), auto_monitor=False)
        self._pv_wfrm = epics.PV("{}Val:Time-Wfrm".format(pv_basename), auto_monitor=False)
        self._pv_wfrm_nord = epics.PV("{}Val:Time-Wfrm.NORD".format(pv_basename), auto_monitor=False)
        self._cb = None
        self._data_is_time = data_is_time
        self.done = True

    def _get_wfrm(self):
        if self._pv_wfrm_n.get():
            return self._pv_wfrm.get(count=int(self._pv_wfrm_nord.get()))
        else:
            return np.array([])

    def kickoff(self):
        self._pv_sel.put(2, wait=True) # Put us in reset mode
        self._pv_rst.put(1, wait=True) # Trigger processing
        self._pv_sel.put(1, wait=True) # Start Buffer
        return self

    @property
    def finished_cb(self):
        return self._cb

    @finished_cb.setter
    def finished_cb(self, cb):
        if self._cb is not None:
            raise RuntimeError("Cannot change the callback")
        if self.done:
            cb()
        else:
            self._cb = cb

    def _finish(self):
        self.ready = True
        if self._cb is not None:
            self._cb()
            self._cb = None

    def collect(self):
        payload = self._get_wfrm()
        if payload.size == 0:
            # We have no data, yeild {}
            ev = {'data': {self._name: 0.0},
                  'timestamps': {self._name: 0.0},
                  'time': ttime.time()}
            yield ev
        for v,i in enumerate(payload):
            if self._data_is_time:
                x = v;
            else:
                x = i;
            ev = {'data': {self._name: x},
                  'timestamps': {self._name: v},
                  'time': v}
            yield ev

    def stop(self):
        self._pv_sel.put(0, wait=True) # Stop Collection

    def describe(self):
        return [{self._name: {'source': self._pv_basename, 'dtype': 'number', 'shape': None}}]

This works with the IOC at:

[stuwilkins]

What should the status object return @tacaswell is this a function which when called returns True if done?

[tacaswell]

In https://github.com/NSLS-II/ophyd/pull/160

[danielballan]

Closing, as the prototype is working with users at CSX. Follow the issue linked by Tom above for more discussion as we merge this into ophyd for reuse across the facility.