search

lsst-dm / dmtn-111

DM Usage in Observatory Operations
https://dmtn-111.lsst.io
Other
0 stars 0 forks source link

For general commenting. #5

Closed ktlim closed 2 years ago

srp3rd commented 5 years ago

A couple of questions:

First, in section 3.1, it states:

"For full-frame processing with ComCam and LSSTCam, an AOS script is expected to control the entire system. It sends “take images” commands via the ScriptQueue CSC to the CCS. the science pixels travel via the Archiver CSC to the Observatory Operations Data Service (OODS), both at the Base. The Archiver sends an “image available” event via SAL that includes the necessary information to form a Butler DataId; this information is also available from the CCS via SAL messages."

I'll note this is talking about ComCam and LSSTCam, not Auxtel, so it appears this "image available" SAL message is not required for AuxTel, is that correct?

As originally specified, the OODS is completely independent of everything. It only knows about the directories it's been asked to observe for new files, and the Gen2 Butler repo. There is no communication mechanism to or from the OODS, so this would be a new feature to be implemented for ComCam/LSSTCam. The OODS would have to send a message of some kind to the ATArchiverCSC in order for the ATArchiverCSC to confirm an image had been successfully ingested.

Currently there's no way to tell that an image was successfully ingested. The OODS directly calls the ingestImages.py command, which always returns exit code 0 on failure to ingest. There is no Python API which can be called directly by the OODS which would: A) allow it to perform the ingest directly, and B) indicate the success or failure of the ingest.

Second, it appears that the services in section 8.2 will not require any "image available" functionality. Is that correct? I say this because it explicitly states "It is not expected that the Commissioning Cluster will be able to communicate via SAL; it is solely for analysis and computation".

timj commented 5 years ago

One comment is that DMTN-133 relies on observing systems knowing that a file has been ingested somewhere in a butler. If that isn't OODS then it has to be some other butler.

Gen 3 ingest should be possible directly from python without needing to fork a python command line.

timj commented 5 years ago

so it appears this "image available" SAL message is not required for AuxTel, is that correct?

If you look at DMTN-133 we really do need AuxTel to be indicating when a file is available in a butler else you are faced with doing logic like:

which seems less than ideal.

ktlim commented 4 years ago

The ImageAvailable event is nominally only for ComCam and LSSTCam, but it will give us flexibility and alternatives to have it present for AuxTel as well. (In particular, it allows us to use the OODS in the Summit AuxTel control loop rather than a further-from-delivery ATCCS-based solution.) Yes, this is a new feature that reduces the independence of the OODS. I'd suggest that the message be RabbitMQ, as that is already in use and publishable from Python. The ATArchiver CSC is one place to receive the message, but we could also start building a separate Telemetry Gateway service to handle things like this. I will look into fixing ingestImages.py to return a useful exit code. Now that I think about it, it is possible that prompt analysis on the Commissioning Cluster described in § 8.2 that is not triggered by the OCS-Controlled Batch service (or replacement) may still want notification of the availability of an image. For now, these may have to be manually triggered or have to wait in a loop polling the Butler.