Should be option for VType json serialization in getDataAtTime

[jembishop]

Hi I am working on trying to integrate the phoebus save restore with the archiver appliance. One of the pain points is that the json format returned by the getDataAtTime endpoint is not ideal. I think it would be good to have option to turn the DBRTimeEvent value into a VType, which can then be serialized by the VType json serialization found here. This serialization can be triggered by an optional url query param, maintaining backwards compatibility. I can implement this if you have no objections. It would involve adding the vtype dependency.

[jacomago]

What about using the protobuf api? It will be more effcient to send data and not require any serialization on the archiver side. Phoebus already has the archiver protobuf as a dependency in phoebus databrowser. Which also does the mapping from archiver events to vtype.

Instead of getDataAtTime you would just use the normal getData endpoint which is wrapped up in https://github.com/ControlSystemStudio/phoebus/blob/master/app/databrowser/src/main/java/org/phoebus/archive/reader/ArchiveReader.java

[jembishop]

Thanks for the suggestion, that looks like a potential way forward. To confirm a url with something like http://archiver.slac.stanford.edu/retrieval/data/getData.json?pv=VPIO%3AIN20%3A111%3AVRAW&from=2012-09-27T08%3A00%3A00.000Z&to=2012-09-27T08%3A00%3A00.000Z will return me the value at that time, it won't just return changes? (it needs to be a snapshot).

Also we would really require that for efficiency reasons it needs to fetch multiple PVs at once (the use case is we have a potentially big collection of PVs we want to fetch values for at a specific time and we don't want to make one request per PV). It seems unclear on the docs whether that is possible, and from this sentence "The EPICS Archiver Appliance has a separate API targeted at getting the value of several PV's as of a point in time" it makes me think it is not.

Thanks

[jacomago]

I did http://archiver_url:17668/retrieval/data/getData.qw?pv=MYPV&from=2023-11-07T12:58:15.000Z&to=2023-11-07T12:58:15.000Z and it gave me one event. If there are no events in the time region it will return the last event recorded before the "from" timestamp. So you will have to filter the events if you want a true snapshot.

Good point, looks like the archiver client library doesn't do multiple PVs yet. You can request data for multiple PVs, The library is at https://github.com/slacmshankar/epicsarchiverap_pbrawclient/, should be possible to add multiple PVs to a query. See https://github.com/slacmshankar/epicsarchiverap/tree/master/src/main/org/epics/archiverappliance/retrieval/client for an example on how to expand the code. The resulting call in the archiver should do something similar to what getDataAtTime does.

It may be worth expanding getDataAtTime to return protobuf events, then adding that endpoint to https://github.com/slacmshankar/epicsarchiverap_pbrawclient/.

Many different ways to solve this problem!

p.s. Another hint, the explicit mapping from archiver protobuf to vtype is done in https://github.com/ControlSystemStudio/phoebus/blob/master/app/databrowser/src/main/java/org/phoebus/archive/reader/appliance/ApplianceValueIterator.java

Maybe you want to refactor some of that code out of databrowser so it can be used in databrowser and save and restore.

[jembishop]

Thanks for your reply, so It looks like you are saying this client functionality does exist here https://github.com/slacmshankar/epicsarchiverap/blob/master/src/main/org/epics/archiverappliance/retrieval/client/DataRetrieval.java#L48 . That seems a good option to use then, instead of save-restore endpoint?