search

lsst-epo / citizen-science-notebooks

A collection Jupyter notebooks that can be used to associate Rubin Science Platform data to a Zooniverse citizen science project.
3 stars 1 forks source link

Removed depr notebooks, pip install citsci package #52

Closed ericdrosas87 closed 1 year ago

ericdrosas87 commented 1 year ago

EPO-8319

ericdrosas87 commented 1 year ago

Removed references to deprecated notebooks and added code to install the "backend code" as a PyPI package

jsv1206 commented 1 year ago

Remove %run Citizen_Science_Install.ipynb in the second cell

jsv1206 commented 1 year ago

utils.run_butler_query is taking much longer time than the version in main branch

ericdrosas87 commented 1 year ago

utils.run_butler_query is taking much longer time than the version in main branch

There weren't any changes to the utils.py in this PR so the delay is likely unrelated.

beckynevin commented 1 year ago

utils.run_butler_query is taking much longer time than the version in main branch

There weren't any changes to the utils.py in this PR so the delay is likely unrelated.

Is this related to Aprajita's comment last week? @ericdrosas87 you seem to have hit the nail on the head with your comment that qserv is to blame.

ericdrosas87 commented 1 year ago

Is this related to Aprajita's comment last week? @ericdrosas87 you seem to have hit the nail on the head with your comment that qserv is to blame.

It could be - the last I heard regarding the QServ outage last week was that an RSP notebook user or users were likely running an unoptimized "naive" query that was eating up all compute resources causing other queries to timeout. It sounded like they didn't have a mechanism in place to identify said user(s) or even confirm that this was indeed the case so I think it's still a bit of mystery.

ericdrosas87 commented 1 year ago

Is this related to Aprajita's comment last week? @ericdrosas87 you seem to have hit the nail on the head with your comment that qserv is to blame.

Oh but if you are specifically asking if this was a "restart the kernel" kinda fix - also it could be. I'd like to give a more definitive answer, but unfortunately I don't have much insight into the RSP Notebook Aspect support world.

jsv1206 commented 1 year ago

Looks like rubin_citsci_core_pipeline.py is missing. I think this needs to be replaced with cit_sci_pipeline

Screenshot 2023-08-17 at 10 25 18 AM

jsv1206 commented 1 year ago

utils.run_butler_query now takes ~15 s for 5 images which I think is reasonable?

ericdrosas87 commented 1 year ago

utils.run_butler_query now takes ~15 s for 5 images which I think is reasonable?

Good to know, nothing from the utils.py interacts with the new backend citSci package or EDC so the processing time isn't affected by this change.

ericdrosas87 commented 1 year ago

Looks like rubin_citsci_core_pipeline.py is missing. I think this needs to be replaced with cit_sci_pipeline

Screenshot 2023-08-17 at 10 25 18 AM

Good catch! I just pushed out an update that explicitly updates the citsci-notebook-core-pipeline PyPI package and also swapped out the object name and tested