[SPRINT] Implementing ELAsTiCC Metrics

[gnarayan]

Implementing ELAsTiCC Metrics

We will have ELAsTiCC Alerts ingested into FAST DB, as well as mock submissions from broker/classifier teams, and we will start to implement SQL/python to write evaluation metrics

Contacts: Time: All Day Main communication channel: #plasticc-public GitHub repo: https://github.com/lsstdesc/elasticc In-person/Virtual/Hybrid: Hybrid Pitch: https://docs.google.com/presentation/d/10jD12cBNtuuph_zO2EAuRWpQLybCl4ALCuudtioYk2U/edit#slide=id.g142601f6059_5_0 Zoom room (if applicable): Use the usual ELAsTiCC zoom - pinned in #plasticc-public

Goals and deliverable

Plots + metrics tables comparing a few mock classifiers on different criterion

Resources and skills needed

Python and ideally some SQL

[aimalz]

@rknop and I had discussed building the ELAsTiCC metrics with proclam. Is this no longer the plan?

[rknop]

Talking to Gautham, the session on Friday is more going to be a workshop for people who want to access the database and pull down the information in order to construct their own metrics rather than building any built-in code for metrics on the TOM itself.

[aimalz]

In addition to updating proclam to Python 3, we ought to make the metrics a little more flexible by separating the per-class averaging from the per-lightcurve metric calculations.

Also, ping me if you want access to the repo and don't already have it!

[mfisherlevine]

I'm sure many/all of you already know this, but just in case you were planning on doing it by hand: py2to3 is a tool for automatically updating python2 to python3. DM did this many years ago, and I can dig out the best-practice instructions for actually running it that we came up with, if that would be helpful.

[gnarayan]

Yeah, 2to3 is fine to start - there will be some numpy/matplotlib calls that have changed, but we should start with 2to3. Re. separating per-lightcurve from per-class - I agree we want this, but this is more than a day. I want to focus on just getting one classifier's results from one broker through fixed up proclam, and that's a good start for now.

[rknop]

Here are the slides I put together as an intro. It has links to URLs for various resources if you're interested in poking into the database of elasticc broker messages and alerts

elasticcsprint_20220805.pdf .

[aimalz]

There is now a DESC fork of proclam which will hopefully make permissions management easier.