lsst-uk / lasair-lsst

Apache License 2.0
0 stars 0 forks source link

Upgrade to DP02 data and schema #89

Closed RoyWilliams closed 10 months ago

RoyWilliams commented 11 months ago

New tools in utility/DP02 for converting DP02 schema (yaml) to AVSC, for pulling DP02 from the RSP, converting to AVRO, and pushing into kafka.

New schema in common/schema with downstream changes for schema browser, for CQL tables

pipeline/ingest.py modified to take new DP02 style data packets. In particular no longer diaSource+[prvSources], rather just [diaSourceList]. Also no diaNonDetections. Also diaForcedSources becomes forcedSourceOnDiaObjects.

Modified pipeline/filter/features to take new name for the forced sources.

Deploy is changed so the settings.SCHEMA is the dp02.avsc