search

lsst-dm / dmtn-111

DM Usage in Observatory Operations
https://dmtn-111.lsst.io
Other
0 stars 0 forks source link

Intermediate needs #4

Open cclaver opened 5 years ago

cclaver commented 5 years ago

This tech note is very useful in describing the end state of the DM deliverables entering operations. However, DMTN-111 omits the needs and use cases for intermediate DM deliverables during the early integration, test and commissioning phases of the Project.

The basic data transport infrastructure (e.g the DM Data Backbone, DM Data Forwarder/Archiver and DM Header Service) is needed in order to complete the planned system early integration tests that are presently underway. We should not be using "rsync" at this point in time to transport date to NCSA.

These system integration tests started with the AuxTel instrument and will continue with ComCam in Tucson - now - with an end-to-end ops rehearsal with ComCam by the end of calendar 2019. ComCam is then planned to be shipped to Chile late February or early March 2020. At which time we will start integrating it into the observatory wide software infrastructure - including DM to transport data from Chile to NCSA.

Does the DM development plan support this system integration schedule?

ktlim commented 4 years ago

Table 1 in the document was meant to illustrate the progression of delivery according to the current development plan. The basic data transport infrastructure is in place in Tucson with the exception of the Data Backbone endpoint that will be available in Chile with the so-called "L1 Handoff" machine.

rsync is a transport mechanism that has a number of nice properties. We may continue to use this transport internal to the Data Backbone. It is acknowledged that cron-based triggering of this transport is not adequate; that will be replaced when the "minimal DBB" comes online.