search

lsst-uk / lasair-project-management

Event handling site for LSST:UK
Apache License 2.0
9 stars 0 forks source link

Lasair Cycle-9 Oct'23 to March'24 #303

Open tms-epcc opened 1 year ago

tms-epcc commented 1 year ago

As taken from Lasair Cycle-9 wiki planning page See also notes from 04/OCT/23 Lasair fortnightly meeting

In order of priority, the Activities, deliverables and milestones agreed as due in this cycle

Note: if any items are left over from previous cycle then these are shown as tickets below

tms-epcc commented 10 months ago

18/OCT/23 Lasair meeting summary

  1. cutouts
    • separate Cassandra systems from the detections - got data into it now looking at retreival
  2. performance evaluation
    • Gareth looking at Kafka, Roy will do ingest module
  3. forced photometry on ZTF
    • SJS wants this for lasair-ztf
    • almost ready
    • will require different plots
      1. Sherlock
    • 150MB file to be copied and run as part of the installation process
    • this is the augmented tables
    • database needs updated first
    • Code update then needs to be coordinated with Gareth's wrapper
  4. Marshall
    • will ask beta-testers to look at this
    • will look at getting a webform or using Github issues
  5. USDF Alert distribution
    • no news on this
  6. Active Watchlist
    • Roy worried about rates so such active resources will need user to confirm continuing need
    • each resource now has a creation and expiry date
    • cron jobs send emails to users about when these expire
    • running on lasair-dev
    • also on lsst prototype
  7. GW alerts (MMA alerts)
    • will begin once outstanding PRs from above items are dealt with
tms-epcc commented 7 months ago

10/JAN/23 Lasair Zenhub summary meeting

  1. Performance evaluation

  2. Forced photometry

    • next step is to merge Dave's code

  3. Marshall notebook

    • Beta test: 6 testers contacted and feedback collected.
    • Notebook is now in Lasair examples
    • Beta test is DONE

  4. Ingesting 6-filter DP0.2

    • this has been done as part of building the test rig for performance testing
tms-epcc commented 6 months ago

31/JAN/24

  1. Put cutouts in Cassandra b. Can delete old cutouts by MJD: no cutouts in DP0.2 so no progress on this
    c. Make and run test to measure speed vs CephFS: now implemented asynchronously and much quicker, see test 2 on Lasair performance wiki page. @gpfrancis notes that there,

    It says explicitly (under test 2) what was previously implicit in the results: that our preliminary tests show Cassandra to be much faster than CephFS at storing cutout images. I don’t think our preliminary testing is very scientific so I wouldn’t put too much store by the exact numbers, but the differences are large enough that I think the conclusion is obvious.

  1. Forced photometry on the ZTF system b. Need to expose on web with new plot: now exposed on the web for ZTF code base BUT this still needs to be done for the LSST codebase (@thespacedoctor and @RoyWilliams to discuss)

  2. Sherlock b. Add DELVE catalogue : need to discuss status with QUB

  3. New alert distribution from USDF: Still waiting for this

  4. Start to build GW alerts infrastructure : now know how to tackle this

tms-epcc commented 6 months ago

14/FEB/24 @RoyWilliams reported that

  1. b) "can delete old cutouts by MJD" DONE hence 1. "Put cutouts in Cassandra" is now all DONE
tms-epcc commented 5 months ago

28/FEB/24