generates basic serviceX queries from dak.necessary_columns
uses dask.annotate to push down data-saving cuts to the query language and remove them from the task graph
This should be sufficient to demonstrate that we can largely automate the use of and delivery-of-data by serviceX such that a one-shot interface suitable for data exploration and full scale analysis is possible.
May or may not be complete by the time of pyHEP.dev - but worth it to talk about.
btovar
commented
1 year ago
Aim - demonstrate a prototype that:
dak.necessary_columnsdask.annotateto push down data-saving cuts to the query language and remove them from the task graphThis should be sufficient to demonstrate that we can largely automate the use of and delivery-of-data by serviceX such that a one-shot interface suitable for data exploration and full scale analysis is possible.
May or may not be complete by the time of pyHEP.dev - but worth it to talk about.