Open CJ-Wright opened 6 years ago
Does the ext
do what you want ? http://nsls-ii.github.io/databroker/whats_new.html#enhancements
Fortunately, databroker already has all the pieces you need to give this a try. Use external_fetchers
(see the release notes -- we have intentionally not documented it further in case we need to rip it out). Then write a function that takes in a Header and does the dereferencing.
Due to a stale browser tab, I didn't see @stuartcampbell's comment until mine posted. We are on the same page. :- )
How do I get that de-referencing when data is going into a callback from the run engine? Edit1: Especially since we might not have a run stop document for a while.
Edit2: Having looked at the source code I'm not certain this does what I need.
The idea here would to have external databases (eg a sample database) which would hand an id and some reference to itself to the databroker and upon request the databroker could replace the id with the actual data from the database.
Our code solves the second part. You have to solve the first part -- shoving the relevant ID into the start document.
I think I can provide the ID to the start document without issues.
This sounds more like wanting an extra EventSource
?
The stop may be None
.
One fun use case of this would be to use a function that when you access the raw data-broker goes and gets a list of all the derived datasets!
We could put ext
into the start document, but that would require squatting one more key.
I don't know if it should be an event source or not, but the rest sounds good.
Would it be possible to attach a ext
factory to the databroker so that every header that came out of that databroker had ext
attached? That way we don't need to take every header and attach ext
externally.
I think what you have described is exactly how ext
already works. Is it? If not, please clarify.
I thought the ext was defined in a per header rather than per broker basis?
The "ext
factory" is attached to the Broker, and thereafter all Headers returned by that Broker have contents in header.ext
. (Try it!)
Would it be possible to support dereferencing external data in start documents?
The idea here would to have external databases (eg a sample database) which would hand an id and some reference to itself to the databroker and upon request the databroker could replace the id with the actual data from the database.
This would be very helpful for at least two use cases: