Support the internal scientists in getting the bulk `queryDatasets` Butler query working to enable 10k data curation in the citSci pipeline

lsst-epo / citizen-science-notebooks

A collection Jupyter notebooks that can be used to associate Rubin Science Platform data to a Zooniverse citizen science project.

3 stars 1 forks source link

Support the internal scientists in getting the bulk `queryDatasets` Butler query working to enable 10k data curation in the citSci pipeline #93

Closed ericdrosas87 closed 5 months ago

ericdrosas87 commented 5 months ago

User story

As the developer of the citSci pipeline, I need to support the internal scientist(s) that work with the EPO citSci team to enable bulk Butler query usage in the citSci notebooks so we can test performance with curating 10k images.

Definition of done

I have been able to support the internal citSci scientist(s) and helped enable bulk Butler queries in the citSci notebook.

ericdrosas87 commented 5 months ago

This was one option for optimizing the way the butler is used in the citSci notebooks, but after meeting with SQuaRE office hours it's clear that this is not a viable option and we won't get any performance gains from switching to Butler.registry.queryDatasets()